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ORGANIZATION OF COW'tlTRR BASED COURSES 
Alfred Bork 

Educational Teclmoloqy Cantor 
University of California 
Irvine, California M717 
(714) 813-6911 

April 26, 19H3 

The computer h,r' been naoa ao a learning device for more 
than a decade, Much use has been experimental, small scale at a 
few Institutions. Now we are beginning to sec more widespread 
use of computers! all Indications are that this rate of Increase 
wM.l continue for a long time, 

One Important ipw feature Is the beginning of commercial 
marketing of compter based learning material, h wide range of 
companies Is entering the fray—textbook publishers, other 
publishers, computer vendors, and new companies developed just 
for this marketing task. Much of the material marketed Is of 
poor quality, based perhaps on the assumption that when so little 
material Is available, almost anything will BelL 

Almost all modules marketed for the computer are "tidbits," 
small bits flttlnq Into courses driven primarily by traditional 
learning modes: lectures and textbooks, The computer units 
prevalent so far assume such conventional courses, At the 
experimental level, however* we find much more extensive 
curriculum with the computer playing an Important role In the 
structure of the course. 

When a new learning mode appears the Initial tendency Is to 
use course structures already successful with earlier learning 
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modes. This Is the naive Initial assumption of a successful 
developer of curriculum material within older learning modes such 
as books and films, But as experience increases, along with 
understanding of the learning capsbllltles of the Computer, now 
: all organizational str ''tr^ courses emerge thr.t were 
not possible without compute. .1b paper delineates 
organizational modes for computer based courses, 

Implicit behind my discussion Is the idea that computers can 
be very effective learning devices, particularly In large 
classes. I will not explore the effectiveness of computers In 
this paper y the reader should consult other sources. 1 

First, I consider general aspects of courses, some concerned 
with learner control and one with a variety of approaches. Then 
I describe course organizations made possibly by computers. 
Content Choices 

Most existing courses In every media assume Instructor 
controlled content, where the Instructor determines everything to 
be learned! students have little choice. Computers, from 
earliest days, have allowed courses where students enjoy 
n f! Dlty in content. For example, the beginning physics 
at Irvine depending heavily on the computer, has allowed, 
ierent variants of the course, ™ two to six separ ate 
tracks through the ten weeks of the quarter, Two sets of 
material are available. The tracks In versions offering more 
choice are built from combinations of these materials. In each 
cas* a fixed set of student materials was available, but th* 
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topics of the dU[p!?nt tracks differed significantly, as evinced 
by the different tents used with, the two sets of water ials.^ 

Kg could cany this further and allow still greater user 
control of content. Some of the staunch supporters believe that 
this is how LOGO should be used. At least in soms cases LOGO 
devotees seem to object to the idea that any fixed curriculum 
should exist at all; rather they suggest that users should make 
all the choices of what to do next* 1 do not believe, however, 
that such completely free student content choice has ever 
occurred in school environments, except on a very small scale. 

So we might thin!: of learner control of content as a 
spectrum , startinq with the usual courses with no control and 
moving through a variety of situations with more ant more control 
on the part of learners. We could argue philosophically for 
various positions in the spectrum, but we will not. As with many 
of the issues raif,c<l in this paper, we need more experience and 
empirical information before we can make reasonable assertions, 
Pace 

Another degree of learner control concerns the pace of the 
course, the rate or r.peed at which students move through th«s 
topics of the course. Most traditional pre-computer courses 
offer little choice; the timing of lectures and dates of 
assignments and exams, establish a fixed instructor-determined 
pace, A student who tries a slower pace is penalized by the 
grading system, and no incentive encourages a faster pace, 

A computer based course or curriculum can allow learner 
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control over pacing. The rate of learning can thus he different 
for different students. For example, a student whose 
mathematical background is indequate for a topic in chemistry can 
take the time to Improve and then can return to chemistry, Or if 
students are a -.ready familiar with a topic, they can move through 
It quickly, 

A possible psychological difficulty with learner control 
over pace is procrastination, It is conceivable that some 
students told to move at their own pace will do nothin ql Whether 
this is a problem or not may depend on other factors in such 
courses, 
Order of Topics 

Just as the usual courses offer learners no choice in 
content oi pace, so these typical courses usually offer learr rs 
no choice either in the order to approach topics, Again the 
fixed order is determined by developers or by instructors, 
Indeed, the behavioral approach to Im-tructional design often 
seems to Imply that this fixed order of topics is essential, with 
the behavioral emphasis on developing a sequential set of 
behavioral objectives. 

But some approaches violate this fixed order and allow 
students to have some measure of control, perhaps even complete 
control, over the ordering of the topics. Gordon Pask champions 
this point oE view, arguing that the best learners do not 
approach the learning task linearly, but rather typically study 
simultaneously several components of the subject area, 

- 4 - 

6 



(instructors often believe utronqly that students cannot or should 
not study Topic B until they have studied Topic A, bo their 
Implicit view Is the Hncnr view just discussed, But this Is not 
necessarily the case for all students. So learner control of 
topics is again a spectrum, varying from no control to complete 
control, 

Ways of learn ing 

Another possibility mny be observed in curriculum 
developments employing the mastery learning ideas, with or 
without computers, The question is, for a given topic, how many 
ways do students have for learning that topic? Often only one 
approach Is paslly available; the textbook presents one way of 
learning the material, and the lecture, while using a different 
mode, often follows closely the textbook approach. So we have 
with textbook and lecture o difference in medkm, but not a 
difference !n approach i students have in most situations only one 
window into the topic being learned, A student studying alone, 
using a book, Is even more restricted. 

In the contrasting situation any one topic or concept 
benefits from a "supermarket" of trays for learning, employing 
different media and different approaches to the topic. Perhaps 
some structured way exists to gather the products students need 
in the supermarket of learning approaches, or perhaps it is left 
entirely to the student. Like all supermarkets, it can be 
relatively sparse or rich with many learning products. 

What is the advantage of supermarket approaches? Benjamin 
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Blum and other supporters of mastery learning ideas suggest that 
different students can best learn in different ways, That Is, an 
approach which works well with one student will not necessarily 
nrk with another student, The claim goes further* For every 
student oome way exists to learn the topic effectively and In 
reasonable time, Even further, some individuals would argue that 
if someone falls to do well in learning something, that person 
did not have the "correct* learning sequence available. The 
implication, then, is that students fall only because those 
students lack adequate curriculum material. This hypothesis 
needs to consider motivational issues, too— It is difficult for 
anyone to learn who Is dead set against it! 

The four factors discussed in the last three sections, fixed 
content versus learner controlled content, fixed pace versus 
variable pace, fixed order versus learner controlled order, and 
single versus multiple ways to learn, can apply in a variety of 
course ordering structures, In moat learning situations today, 
the choices are (see Figure 1) for the small end left, of each 
spectrum; little learner choice is offered, On the other hand, 
choices with too much freedom may be impractical for many 
learners, 

Readers should consider each of them in the light of the 
next major section of the paper, which considers ways of 
organizing courses, 
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Co urse Organizations for Computer Based Courses 
1. Traditional Course with Computers 

The first mode of organization is, by far, most prevalent 
when computers are used today. The organization of the course is 
like most non-computer courses, with the same fixed sequence of 
topics and with a fixed pace for all students. The computer 
serves an important part of the instructional process, but the 
organization of the course remains traditional. 

The computer may replace or augment the book or the lecture. 
In either case the computer material will, if well designed, be 
much more interactive than the book or the lecture, The computer 
is simply replacing one or more of the learning modes of a 
traditional course, with the rest of the course structure 
remaining intact, Courses of this kind are fixed content, fixed 
order courses, with no learner control, limited in the variety of 
learning modes and approaches offered to students, Testing 
occurs infrequently at periods of weeks, 
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In this second structure for course organization we see the 
computer with a course structure popular on a small scale without 
computers. This structure Is called the personalized system of 
Instruction, the Keller plan, or the mastery learning approach . 

The basic structure of the Keller plan breaks the course 
down Into units, typically requiring about one week of study for 
an average student. For each unit the course description 
Indicates what the unit covers, what learning resources are 
available, and perhaps contains a sample test. Students study 
the unit using available learning resources! the choice oE 
resources may be left to the student, but many courses of this 
type stress the book as the principal learning medium. When 
ready students take a test covering the unit. The test is graded 
Immediately, and the student receives feedback about difficulties 
without delay. To continue to the next unit, students must 
perform almost perfectly on the unit test; otherwise additional 
study Is suggested before another version of the test 1b given, 
In some cases additional study is required. Grades' are assigned 
on the basis of competency rather than on a normative or 
comparison basis. 

The course is usually student paced, students having partial 
or total control over how fast they traverse the material. 
However, In most Keller plan courses students must face the end 
of some designated time period (quarter, semester). We can find 
many variants to th Is structure, and experts might disagree as to 
which were "really* Keller plan or PSI courses. 
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A computer-deponent Keller plan course can use the computer 
In either one of two w:;j| or In both. The computer can be used 
In instructional mode, accessed by students to learn the topics 
of the unit. Computer material le always available, unlike 
lectures, allowing the course to be more self-paced* This 
learning role of the computer is like that In traditionally 
organized courses; the same material may serve in both, junt as 
the same book may serve both traditional and Keller plan courses. 

tye other way to use the computer in Keller plan courses Is 
for the unit tests . These test, e fundamental in the course, 
telling both students and instructor when the student has 
mastered a unit. This testing function can be partially or 
entirely on computer. 

I will not give a full description of computer based 
testing, 3 b u t readers should understand that, such testing can be 
very versatile, the problems can be selected from pools d items 
or, often better, front sophisticated problem generators which 
produce a widely varying collection of problems of a given type. 
Tests can be devised so no test is ever given twice under any 
circumstances. 

Immediate feedback on correctness can be offered. We can 
also provide significant aid to studentB, aid very much tailored 
to students 1 own explicitly manifested problems. 

The introductory pre-calculufl math course at the University 
of California, Irvine, developed by Stephen Franklin, uses on- 
line tests without built-in aid. It assumes that Instructors or 
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tutors will provide this aid. On the other hand, the tests In 
the Introductory physics course at Irvine incorporate extensive 
built-in aid. 



Unit 
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\ resources / — test /■ >- 
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Figure 3 - PSI Structure 



the learning takes place during testing. One variant of this 
idea, exploited by Kenneth Bowles at the University of 
California, San Diego, in his Pascal course, allows students 
access to the tests learning sequences before they take them 
as tests. 

With this method we are approaching an organization not 
possible without the computer. Tests combining learning are 
practical in large classes only if they are available on the 
computer. He will see other examples of courses demanding the 
computer In some, but not all, of the later organizational 
structures discussed. 



3* Test Driven K ell er Plan Course 

The notion that computer based tests can contain a component 
of l earning as well. *s testing, suggests a variant in the usual 
Keller plan format. The notion is to abandon much ot all of the 
preliminary learning material of each unit and focus learning 
primarily within the tests themselves. 

Although teachers commonly tell their students that tests 
are a learning experience, students seldom believe it. But 
evaluative studies of our introductory physics course indicate 
that 80 to 90 percent of the students Identify the tests as the 
major source of foaming material of the course. Other learning 
material may be available, such as a text and lectures, in such a 
course structure, hut the burden of learning has shifted) much of 
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Figure 4 - Teat-driven Keller Phn 



*• Learning Cycle 

Another method of organizing curriculum development, with 
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distinct featuren qolng boyond those already discussed, has been 
proposed by Robert Karplus, University of California, Berkeley. 
This method ts derived from Karplus'A study of Jean Plaget's 
work. 

The learning cycle approach emphasizes a notion seldom 
stressed in conventional courses! an experiential basis is an 
important component of the learning process. The importance of 
experience relevant to later learning Iq recognized with young 
children, with the use of manlpulatables in teaching mathematics, 
but plays a lesser role In more advanced courses. A typical 
university course does not even attempt to provide any such 
experiential basis, notwithstanding this being our best hope for 
developing student*?' intuitive grasp of the subject area. 
Developing students' Insight is just as Important as developing 
ability to do the e*n»ntiai tasks. 

Karplua 1 concept of the learning cycle involves three 
components, the first being the experiential component, the 
second a more formal learning component. The aim of the third 
component Is to see whether students can use what they have 
learned. 

Computers can phy a role In all or some of these 
activities. Thus, In many disciplines computers can furnish, by 
means of computer simulations, the experiential first component 
of Karplus' learning cycle. These computer based simulations 
provide students a variety of experiences, either simulating 
phenomena In the real world or creating phenomena that could 
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never be found In the world. Within the Educational Technology 
Center we call such activities "controllable worlds," The tern 
"micro worlds" le also used. For example, for the beginning 
physics course at Irvine, mentioned here, we have available an 
F - ma Newtonian worldi students can "throw" bodies and watch how 
they behave under the action of various force laws, All aspects 
of initial conditions, force laws, constants In the force laws, 
scale of plotting, what variables are plotted, or combinations of 
variables, are completely under students' control.* 

Computers can also function In other components of learning. 
The entire learning cycle can be computer based, for each unit of 
a course, As an eiample, we could consider materials developed 
at the Educational Technology Center which follow such an 
organizational scheme. One project concerned public 
understanding of science, stressing the nature of a theory, how 
theories are discovered, and related Items. Another project 
focused upon helping students to reason formally, In the Plaget 
sense. Bach product has an organizational structure similar to 
the learning cycle, beginning with an experiential basis. The 
various modules developed in these projects are discussed In 
fuller detail In the literature. 5 
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Figure 5 - The Learning Cycle 
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5. rnm plete Mastery Learning Cycle 

The organizational scheme suggested In this section combines 
the features of the last three schemes mentioned. It employs 
something akin to Karplus' learning cycle, particularly with its 
em phasis on the experiential basis. It also incorporates the 
mastery learning ideas of PSI or Keller plan courses, requiring 
students to learn a topic almost perfectly before leaving the 
material. The computet is suitable for all parts. Self-pacing 
1, typical, and it in easy to allow alternate contents. Again, 
the type of organizational structure that would be possible, wi 
computet simulation and mastery based testing, would only be 
possible with computer modes of organizing courses. 

Recently the armed forces have discussed a mode of course 
organization called functional context training, which starts 
with the application, perhaps simulated on the computer and then 
shifts into learning-sequences, depending on students' 
difficulties In the application. This procedure closely relates 
to the complete nashty learning cycle, particularly if the 
Initial activities ate simulated o.< the computer. 

Very few course follow the compete mastery learning cycle. 
But the promise of such courses is considerable. This approach 
deserves extensive further study. 
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Student Or gani c Course s 

This possibility, already suggested In connection with use 
of LOGO, provides students a very effective set of tools on 
computers for the turning process, Including simulations and 
languages. Student?' are allowed virtually completer or mostly 
complete, choice as to which of these tools they use and how they 
use them. The Instiuctors Impose no ordering on the couise, at 
least in the ideal r^nse. Students will presumably learn If the 
environment In sufficiently rich* 

As mentioned, it is not clear whether a fully student- 
organized course has ever been tried In schools, except on a very 
small scale. I find it difficult to see how lifelong learning 
experiences could bo organized In this fashion; but this approach 
certainly presents interesting possibilities, for certain areas 
and for certain students, worth further experimental study. 
Concluding Remarks 

Undoubtedly courses heavily dependent on computers could he 
organized In other ways. The suggestions In this paper seem 
promising and all rtasecve further exploration. The Issue of what 
Is "best* In each situation can only be determined by empirical 
investigation. 

One light raise a further question. In this paper the word 
"course* appears frequently. But do we need courses? If we 
think, for example, of typical schools or universities, the 
existence of courser,, particularly In universities, Is primarily 
for the convenience of the registrar or perhaps the Instructor, 



butnot for the convenience of students. With computers we can 
envision learning as a continuing process, without our current 
artificial divisions and restrictions. 
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Basic Mathematics In Colleges anJ Universities - 
Computers as a Solution 

Alfred Bork 

Educational Technology Center 
University of California 
Irvine, California 92717 
(714) 833-7452 

June '"2, 1983 

Students at every level of education enter with a variety of 
knowledge derived from previous courses and other modes. 
Typically universities are very poor In dealing with this 
situation of heterogeneous background. Furthermore, just looking 
at transcripts offers no clues. Mathematics books are 
particula ly Important because both math and science courses 
depend on them, Many universities require examinations for 
entrance into such courses as calculus and have courses prepared 
for students below that level. The problems of students are 
seldom those that can be handled well in a "whole remedial 
course" procedure. Each student has unique needs depending on 
that student's background. While in universities we give lip 
service to treating students individually, very little of this 
happens In our classes. 

As a result, during the past few years we have had "inner 
universities" arise within many universities* This inner 
university Is called a "learning resource center" or soma similar 
name. It is a court of last refuge for students In trouble In 
their courses. The existence of learning resource centers Is a 
strong condemnation of our traditional courses, If we were doing 
our job fully, such centers need not exist. 
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I will argue that a clear way of dealing with this situation 
exists which, if, carried out nationally, could solve the problem 
almost simultaneously and at a reasonable cost through the entire 
country* I refer to the development of flexible, highly 
interactive, graphic computer based learning modules to cover 
these areas of basic math. This material would be used by 
students strictly on their own, or it could be available in 
learning resource centers. 

In those universities moving toward requiring all students 
to have certain computers, such as Carnegle-Heilon University, it 
would naturally run on those computers. We would expect It to be 
available on a range of common computers, those coming into use 
In schools at present, No credit or course ty,,e structure is 
needed with this material, as the units would be directly usable 
by all students, Some conventionally organized universities 
would use them within courses too. The emphasis would be on 
mastery learning, demonstrating conclusively that the student 
fully knows each of the mathematical tasks Involved, 

The basic format of these materials would be a series of 
problems, covering the entire range from simple arithmetic 
through algebra and trigonometry. The problems would all be 
presented at the computer. Each problem would arise .out of a 
computer based problem generator , so an infinite number of 
similar problems would be available to students, Some immediate 
assistance would be available to students in connection with each 
problem) this assistance would be hi'Iily interactive computer 
aid, not just verbal information which the student reads 
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passively. Assistance would also be available on a broader scale 
for students who are having severe problems with certain 
Identifiable types of problems, Students would then be recycled 
through the problems, with additional aid if necessary, until 
tjjey can perform at a mastery level on each type of problem, 

Note that in material of this kind students move very 
quickly'over anything they know, and so all student effort Is 
focused on exactly the material they need help with. So student 
time is used in an effecient manner. The materials vould.also be 
flexible; particular probbms could be taken out of the pool or 
other changes made in the way the exams run to suit individual 
circumstances. . 

The materials would follow the best strategies available for 
producing modern computer based learning materials*! Extremely 
competent teachers from all over the country would be Involved in 
the pedagogical design sessions to structure the material, 
Additional information about production strategies Is available 
from the Educational Technology Center. * 
Costs 5 

Although no careful cost estimates have been made, 
comparisons of this project with other efforts at producing very 
high quality learning material on the computer would suggest a 
budget of about $350,000; two to three years would be required. 
It is expensive to produce good curriculum material, independent 
of the medium. We could not expect these materials to be 
produced by one teacher in that teacher's spare time, 

Whether we regard these costs as considerable or as small 
- J - 



depends on the scale being considered. That is, If the materials 
were used in only one or two universities, the cost would be 
great l i^4Hhe materials were available throughout the entire 
United States or even in other countries, then the cost would be 
small on a per school basis, 

A number of possibilities for' funding can be suggested, 

1. Federal Grant 

The simplest way to fund a project of this kind would be 
through a t federal grant. 

2. Commercial Support 

A second possibility would be support from a commercial 
publisher or computer vendor, with the idea of marketing the * 
material. 

3. University Consortium 

A third possibility would be a group of universities working 
together! each contributing some funds to the total project, 
These universities presumably would then* have free use of the 
materialj except for duplication costs, and they could market It 
to other institutions. They might well make a protit eventually, 

4. Combination Funding 

Although it complicates funding possibilities, funding might 
come through some combination of the Above sources. 
Final Comment 

Adequate material of this kind could solve the problem for 
most universities in the United States, providing an extremely 
flexible way of meeting the needs of students and one which 
conserves valuable student time, I frankly see no other 
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possibility. / ' t> 
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Ahn tract 

Computer baned lonmlng natn Inln wywldnly In 1 ho I r 
apparent "IntnlUqniim/ At one extreme, mnny proqrnmn currently 
belnq Rnlct Tor Hip educational mnrkrt are extraordinarily limited 
(add unnoceMarlly no, we contend! In llieli ability in renpond lo 
natural, open-emM oner Input, At Hie other ok I. mine, Iherr ban 
been much dUcuHon recently about Ro-ralN "iiilelllqenl: 
tutorn" Hint \m techniques from artificial Inlelllqence, but at 
a considerably graeter corrt In terms o( hardware requirements, 
This paper onen three examploa of learning material's for pnrnnn.il 
computers, produced with careful attention to podaqnqlcal design, 
that appear to be "Intelligent* but which have been Implemented 
on Binall machined and with a minimum amount of software overhead. 
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The past few years have seen Increasing interest in and use 
of the computer as a learning device in a variety of areas where 
the computer Is not the subject matter of the area, Thus, we see 
It being used to aid in learning elementary arithmetic, college 
physics, and in many other disciplines, Indications are that the 
computer's presence in he educational situation will continue to 
grow* The sudden emergence of a dozen or so magazines 
particularly concerned with computers in education and the entry 
of at least a dozen publishers (both older, conventional, book 
publishers and new companies) into t.iio market are indications 
that the movement has gained momentum. 

Unfortunately the quality of many of the commercially 
available computer based learning units, leaves much to be 
desired, Many of the examples (commonly available) do not 
reflect the best usage of the computer. 

This situation may give some Individuals the impression that 
computer based learning material always deals with trivial or 
unimportant learning activities. The existing commercial 
examples have tended to reinforce this notion. Another point of 
view is that existing examples Indicate that computet based 
learning is still extremely poor, but that a future type of 
activity, often called "intelligent computer based learning/' 
will lead to better material. The claim is that intelligent 
material, generated by the techniques developed In artificial 
intelligence, will be required to get away from the trivia. 

My Intent in to argue through examples that this is not the 
case. That is, we will argue that we already know how to produce 
good computer Lased learning material) indeed, we will point to 
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many examples of material which are, we think, effective learning 
materials. This is not to say that we cannot Improve through the 
use of Artificial Intelligence or a variety of other techniques. 
Rather, it Is to say that it Is a gross oversimplification to 
claim that we have not developed good material without these *; 
techniques, 

The principal aim of this paper is to contribute an 
understanding of how the computer can ha effectively used in 
education. There is already ample evidence to show that the 
computer can be effective in a variety of situations. 

The three examples we will use show a variety of strategies 
in education, They come from work done at the Educational 
Technology Center at the University of California, Irvine. We 
have been involved in the development of computer based learning 
materials at Irvine for fourteen years and have developed In many 
areas and at many levels, The three examples are by no means our 
total output but were chosen to show our range. 

The implications extend far beyond education. People in 
many areas besides learning desire Interactive computer based 
material that is friendly to use. For example, the entire office 
systems field could profit greatly using many of the tactics used 
here, both in introducing people to the new environment and in 
coaching them through the new environment. Many applications of 
the computer involving novices could also use the types of 
techniques which have been used in good computer based learning 
material. For example, it Is often necessary for people with 
little computer experience to access data bases. Yet many data 
bases are currently designed in a way that makes this access 
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pxltemely dlfflrult. Tim bthl toqr nphlc data bnnpn r for example, 
were Initially touted an helnq nnnhln hy the casual unerj 
however, most of the bibliographic data bason are In (net uned by 
trained Ubrailans, 
Example i - Physics Quizzes 

The first set of material to bp discussed wan developed In 
connection with an Introductory physics coursn at the University 
of California! Irvine. This coutflp has been tauqht for 
approxlwatnly eight years In an Increasingly routine fashion. 

•'he basic pedagogical problem wns to provide Individualized 
aid and assistance to tlie many students who take a large, 
beginning sconce course. The computer has proven to be an 
effctlvo and popular medium for achieving this. One variant of 
the couron has \wd the computer to provide such aid, In recent 
yr>ars In which this course was taught , with the computer as an 
option, about 400 students chose the computer variant) less than 
half this number chose the noncomputer variant. In universities 
large numbers of students learn through lectures and textbooks In 
an environment which provides extremely little Individualized 
aid. 

A second goal was that all students learn everything well. 
That Is, we did not want to create the type of course where some 
students learn the material well, but many students, perhaps the 
major Ity, ( end with only a very ^partial view of beginning physics. 
This type of course Is often called a mastery course, because the 
requirement Is that every student master the material. This Is 
not- to say that people: will not leave the course for one reason 
or another} our hope was to reduce the number of dropouts over 
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the standard cnursp.Thln course, which ban \w\ well described In 
the llteralum,' will only he described hr lofty limn, The hump 
In divided Into a sorles of units, nlmllar (o the nynlpm used in 
the Personalized System of Instruction ;h Knl lor Plan nttatpgy. 
For each unit never al on-line tests arn available. S Indents mint, 
do almost perfectly on these tests befo'e being allovpd to 
proceed to the next unit. Typically the student will take a 
giver test n number of times. The tests are not only tests? 
rather, they provide large anounts of learning maleilal. Often 
this learning material is highly specialized to the precise 
student needs. That In, If a student works a problem, the type 
of help given to the student who does not succeed may reflect the 
specific difficulty the student was having, Although thin 
material Is described as quizzes, there Is a constant flow bark 
and forth between the quizzing and the learning activities In the 
material. We should comment, Incidentally, we never u:>e such 
undesirable techniques as multiple choice within these quizzes. 
Tho student completes the course by completing the units In the 
course. Tiierc Is a separate written final exam, 

In order to give a better understanding of what Is Involved, 
we will describe one of the 27 on-line quizzes, a quiz called 
SLOPE. This quiz Is taken early In the conventional course. 
(Students have a choice of two different courses, to allow 
student choice of content.) SLOPE will normally be neon hy the 
student In the first we( . oi *:he course. It is of moderate 
difficulty, from a content point of view It might be described 
as geometrical calculus, but tire co^ext of the physical 
concepts of position, velocity, and acceleration, the aim Is that 
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students given curves of one of these versus time should be able 
lo identify and construct correct curves o£ the other two, Thus, 
differentiation and integration are involved, but with graphs 
lather than with functions. 

SLOPE begins, as do many of the quizzes, by asking students 
if they would like to have a help sequence for Lhc material, The 
help sequence does not count as part of the 40 minutes that 
students are given to complete this add the other quizzes, A 
help sequence is an interactive computer based learning sequence 
going over the concepts of the quiz, Within the help sequence, 
very different from the quiz Itself, students proceed to 
construct a velocity-tine curve, point by point, from a position- 
time curve. Remember that It 1b the student's option to use this 
help sequence. 

The first step In the quiz consists of several questions 
which check on the student's knowledge of how to read graphs, 
While we could expect most of our beginning students to be able 
to do this with no difficulty, it Is important in the early 
stages of a physics course to identify students who are lacking 
such critical skills* The first question, for example, presents 
a curve (a 'different curve each time the program is run) and asks 
the student what the position is at a randomly selected time. 
Note that In effect we have an Infinite variety of questions of 
this kind, and that therefore no students ever receive an 
identical question, 

In the second stage we try to find out whether students can 
identify points where the velocity is 0 from a position-time 
curve. The Internal pointing device in the system is used for 



Alfred Dork e 

this purpose. Again, the help is very selective, Instructors 
will often know what the common student difficultien are, and so 
these can be anticipated within the quiz, For example, on this 
problem the most common difficulty Is that students will point to 
places on the curve where the position rather than the velocity 
Is 0. In this case it is easy to give very responsive and 
helpful feedback, as we do. 

The r <ain body of SLOPE has three parts, First, the student 
must be able to identify a correct velocity-time curve, given a 
position-time curve* This is repeated for an acceleration-time 
curve, given a velocity-time cures. Finally, in the last part 
the student Is given a velocity-time curve and Is asked for 
numerical information about position at certain times. Again, In 
all these cases random choices are made, fio the questions are 
different each time the quiz is run. 

We remind you that SLOPE is only one of twenty-seven 
quizzes* These quizzes were developed mny years ago in a 
timesharing environment. Currently we are beginning the process 
of moving sixteen of these quizzes to personal computers, where 
they will be marketed by a major textbook publisher. 
Example 2 • A Controllablejorld 

The second example presented here is also Intended for the 
first part of a beginning physics course. The pedagogical 
purposes, and therefore the computer strategies followed, are 
entirely different. The Issue here Is an important one in any 
teaching area. It is relatively easy to teach the techniques, 
and even to teach people how to solve particular kinds of 
problems. But the development of insight and intuition into 
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difficult object matter ar?an U vnry difficult to teach 
students, and so is frequently Ignored, 

The program described In Intended to build Intuition In 
beginning physics, whore students arp looking at systems 
Involving moving bodies. The Interesting lnsues are the forces 
Involved, the parameters In the forces, and the Initial 
conditions. Programs of Mils kind are possihle In a variety of 
areas, He call them controllable worlds. They have also been 
referred to an mlcroworlds by Hie computer education group at the 
' Massachusetts Institute of Technology. In any cane the notion If? 
to provide just those capabilities by means of a facility which 
allow the ntudent to play with the phenomena Involved, to gather 
experience through either a free play arrangement or, better with 
large classes, a play arrangement where none guidance Is 
provided. 

A particular program was first developed in a timesharing 
environment. Then another variant wag developed several years 
ago In a personal computer environment. The two programs are by 
no imam identical, Each had some capabilities not available to 
I he other, but they each had the same basic capabilities, They 
were both very easy for the novice student to use and had a great 
deal of Inntructlcr.dl material on how to use the program built 
Into the program, Therefore, It was not necessary to "teach" 
people how to use the material.. A wide variety of responses was 
accepted. If one wanted to draw a graph, one could say draw, 
graph, or plot, for example, and the arguments could be given In 
various ways too, such" as "i, y\ or'vx vs. x," The student 
can plot any two or three meaningful physical variables, Indeed, 



In the personal computer version the things to be plolM ran be 
any algebraic combination of any of the variables. Thn student 
also liar: control ovet the Initial conditions, the parameters In 
the force laws, and on what force law In used, All therm, 
however, are not required Inltlally-whon Hie student first picks 
a force law the program In Immediately In a pnsllicu to plot 
flow tiling. 

In a relatively brief time the student can acquire many 
experiences relative to moving objects, can vary parameters to 
see what happens to the graphs. Students are not restricted to 
looking at the motion of the objects, but can also plot the 
various abstract spaces which give Insight into I he behavior of 
physical systems, 

Mien they were first developed these programs had some 
severe problems in actual usage. While highly motivated students 
In class would use them eagerly and enjoy them just as the 
professionals did, many students were not successful. He 
overcame this problem by developing a set of workbook materials 
which guide students Into the critical cases and which nmtrol, 
to a certain extent, the. discovery process. The student wan 
always free to go off on his or her own If that Interested them. 

He expect the personal computer w**nlnn of this program, 
with the associated workbook mater 1 -* bo Available soon 
throuqh CONDUIT. 
Example 3 - S cie ntific Literacy 

The problem raised by the third r Maple Is an entirely 
different one with an entirely different age group. He are 
interested here h a very wide age group. The programs developed 
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In this project, concerned with public understanding of science/ 
have worked with students as young as nine years old. They have 
also worked with, senior citizens. The programs were developed 
ant) tested primarily for public library environments; assuming 
that anyone who walked into the library, <ith no computer or 
subject matter experience, could Immediately learn from the 
programs, without benefit of text materials or teachers. , 

Note that the problem is again a serious one. Many people 
in our society have little understanding of the nature of 
scientific activity. While they sometimes use the vocabulary of 
science and are, in the political process, oEtefi called upon to 
pake dccisioiis which depend on some understanding of science, 
this understanding is often very weak. Schools do not help in 
the matter. Host science programs in schools tend to give a 
quite inaccurate view of how the scientist goes about discovering 
theories and, indeed, what the theories look like. This is true 
at the elementary school, at the secondary school, and it is even 
true at university level science courses, In fact, the topic of 
the nature of scientific knowledge is such a difficult topic that 
it is seldom treated in conventional courses. Hence, it seems to 
us to be an important problem. He felt that the computer could 
do things that were very rarely being done in our current 
educational system. 

We developed a number of modules in this project, each about 
one and a half to two hours long for the average student. The 
basic tactic was to put students in situations where they had to 
behave something like a'sclentist behaves, That is, the students 
gather evidence, form hypotheses, test the hypotheses and change 



them, until they arrive at a satisfactory theory. In some cases, 
as with the one described here, the situations are real. In 
other cases the situations are imaginary. The environment is a 
friendly, helpful environment. A student will not flounder 
forever trying to do something If he or she has difficulties. 
The environment is a discovery environment in which a 6tudent who 
is making no progress at all will begin to get midges in the 
right direction through built-in tutoring, done unobtrusively. 
Indeed, with all students we try to maintain the feeling of 
success, the notion that they are succeeding in the learning 
process, 

The particular program we will mention is one that concerns 
batteries and bulbs. During several hours of the session the 
student develops a model for simple electrical circuits involving 
batteries and various light bulbs. The student is not told the 
'"citical concept as Is usually the case in the typical classroom 
presentation. Rather, the concepts are discovered in a fashion 
just described that is quite consistent with the way a scientist 
might go about investigating such a problem. 

The reader who Is familiar with some of the curriculum, 
development projects in the United States in the 1960*3 will 
recognize the batteries and bulbs materials described, because 
similar nni ? n vr v vol oped in a ' ,j r of the elementary 

* cul: vj cts. But as developed then, 

t li ..») In ideas similar \ ne ones expressed 

heie, nave not woii.d it ass situations. The problem has been 
that the units were highly teacher-dependent, and in general 
teachers are not willing to spend the time and effort lb maintain 
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the rqiilpuirnt nnd to work with Individual studonh Li a discovery 
mode, Thenn materials luve been little U8i?il despite the Lirqc 
wiml.s of money which wont inhn thorn because Miey do not have a 
qo»n1 "Impedance" ma Wr with the claanroom situation. The 
batteries and bulbs material in, an seen by out testing In the 
pnl'Uc library environment, Independent of teachers and could he 
uned by students all over i.he country. We are currently 
Itfginnlnq discussions about marketing this fflatorlal with a number 
of possible vendors. 
Cone lus ions 

The three examples presented do not represent the full range 
of different ways the coiputer can he used in education. But 
they do show three different strategies, and in each case they 
show that the computet even with current tactics can play a very 
important role in dealing with critical pedagogical problems. In 
no case are th| problems oelng faced trivial. Indeed, In, each 
case we did nJt start by asking "How can we use the computer?,' 
but rather wel started by paying "How can we help students who 
have this difficult learning problem?" 
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ee both common factonVand differences In these 



programs. First, note that the computer, environments run from 



timesharing 



through personal computers. More recently our work 



: In the Educational Technology Center has concentrated on personal 
computers, and we believe that this Is the most likely future 
direction tk computer based Instructional material. 

One thing that in not obvious from the descriptions so far 
U that alt these programs are very g^ohlcal. That Is, they all 
use visual li.formatlon as an Important component of the learning 



process. The notion of developing learning mnt«Hal wlllmul the 
use of visual Information does not reem hi us to hn denlralite. 

Ml the programs are highly Interactive. Klieii nhdnnis inn 
them they have the feeling that they ate dealing with an 
Intelligent program, nut the Intelligence does not one f.-m the 
particular techniques which have been (Sloped in artificial 
intelligence, but rather from the fart, that In each case Hip 
programs have been developed by groups of highly experienced 
teachers. It Is the experience of. those teachers that Is 
reflected in the program as Intelligence, and It Is this that 
lends students to comment that the programs are highly responsive 
to what they type In and to their Individual needs. 

Thus, the programs appear tp be "Intelligent" from the user 
point of view, In spite of the fact that the compiler scientist 
might not view them In this fashion. The key Is the production 
process which, in early stages,' involves small gioups of 
extremely competent hnchers and gives those Individuals maximum 
freedom to use Insights from their teaching experience, flio 
notion of working in groups Is an Important one that needs to be 
stressed herei we find that no one Individual ran produce this 
mater IbI as effectively as a group can. The production process 
has been described lully In literature available from the 
Educational Technology Center. He believe that ft Is one of the 
key Ideas to future computer based learning material. 

It should be noted that. all three of l.he S e programs try to 
exploit the medium of the computer as fully as possible, not 
necessarily bringing In other learning media. Thin Is 
undoubtedly realized most successfully in the scientific literacy 
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material, which normally runs In situations in which no teachers 
and no print material and no films arc available. This is true 
to a lesser extent with the physics quizzes. With the 
control lahle world program;), as indicated, we have found that the 
print material is useful, bul there are tactics where it can be 
avoided here too. Projects vhich depend on extremely complex 
mixtures of learning media seem to me to lead to problems in 
typical classroom environments. 

Finally, we wish to make it clear that we am not arguing 
a gainst the use of artificial intelligence in computer based 
learning material. Indeed, we think that those techniques will 
also be extremely valuable in improving the quality of learning 
material. But we think that what is needed is some combination 
and hybrid of the best aspects of contemporary computer based 
learning and the best future possibilities that may come from 
built-in Intelligence. Again, it is not an either/or question 
and phrasing it in that fashion creates an artificial dilemma. 
The issues we need to deal with are learning issues; the problems 
we need to deal with are the important learning problems. All 
computer techniques that contribute to this should be uped . 
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A preliminary taxonomy of ways of 
displaying text on screens 

Alfred Bork 

University of California, Irving 



This paptr considers the amputee screen as a spat* 
time arena. It develops a taxonomy of the various 
variables associated with the display of textual 
material 
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Introduction 

More and more people are seeing text material 
displayed on screens through both video and 
computers. For example, text«c*sed display* are 
often used for reference, perhaps interactrrely, at 
with reservation clerks ax I'^vei agencies or airlines, 
who use such displays for many hours each day. 
Similarly, in a computer-based learning situation 
the student may use a screen for an interactive 
program or in the near future as a surrogate book, 
ptnscularly as the coat of delirering textual 
material via computer or video technology declines 
as compared to the coat of delivering it on paper. 
The question of how text will be displayed oo the 
screens will become particularly important as 
videodisc technology and computer technology are 
combined. 

In spite of this increased use of screens for 
displaying texts, the vast majority of screen displays 
are probably far less effective than they could be. 
The issues involved include legibility, memorability, 
and motivation. Increasing a display's effectiveness 
is not just a matter of improving the screen's 
resolution or changing from monochrome to colon 
ideally, w* would also take account of users' reading 



purposes and draw upon the skills of information 
designers in deciding how information should be 
presented. We could then see the possibility oi; 
siding and improving the reading process, a* in 
aueraaive reading; a new reading styta possible only 
with the new media. At present, we are rather far 
from this. Often, the way text appears oo a screen 
is almost an accident, determined either by the 
progr am ming languages involved, by the computer 
hardware, or by limiting users' control to matters 
that have nothing to do with the visual appearance ' 
of text. This is perhaps excusable, since little is 
available in the literature as yet in the way of good, 
empirical evidence on how to display text oo a 
screen; further, there is a scarcity of in f or m a t ion 
designers with experience of the new media, so 
there is a danger that designers may rely on 
experience with older media, such aa print, and 
carry over inappropriate solutions. 

In our view, all those who work with new media 
are in urgent need of an appropriate c l assification 
trrrm* The body of this paper presents \ 
preliminary attempt' to draw op such a s c heme , in 
the form of a taxonomy of the various types of 
textual treatment which are now possible in '.be 
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computer environment. This may provide s better 
basis than is currently available for the information 
designer who has to compare and choose between 
various presentational options. 
Display mtdhtm versus print material 
Many information providers treat the screen as if 
it were the page of a book. This is not surprising 
as the book and related print forms are the dominant 
reading modes now. Yet even a cursory examination 
of the display screen and reflection about the 
possibilities indicates, that many differences exist. 

The quality of tax possible oo most CRT screens 
is much poorer than the quality of text available 
in print materials. Letters formed from s limited 
dot matrix are at best crude. Hence, legibility is 
Likely to be an even more important issue on the 
screen than it ia in print. 

The possibility of the control of the time domain 
ia another striking difference between screen and 
print. Many different time considerations are 
possible. In this regard the screen has a closer 
affinity to a film than a book. 

Furthermore, the economics of the two situations 
are very different. One of ;bc most striking 
differences comes with the question of blank space. 
In boohs space costs money, since the cost of printing 
books is roughly s linear function of the number 
of psges in the book. However, on the computer 
screen blank space ia free. Hence, size for size, blank 
space can be used in a much freer fashion in 
computer displays than it can be in the equivalently- 
sized printed medium. 

I do noi wish to imply thst all the previous work 
done with text display in printed form is useless 
for understanding what happena on (he screen. In 
fact, much of the material concerning legibility and 
readability appears to be directly applicable. But 
without further work we cannot know if this 
information extends to the tcreer. And, as 
indicated, there are many differences between print 
and screen. 



The classification scheme 

The purpose of this present paper is not to describe 
experimental work showing how the screen should 
be used. Rather, it is a preliminary attempt leading 
in that direction, an attempt at developing a 
taxonomy of the various types of textual treatment 
which are already possible in the computer 
environment. Below, I consider both spatial and 
temporal factor*. 

The reader should realize that many of the factors 
discussed could be controlled either by the designer 
or the user. Thus, rhe user might idjust timing 
delays to suit individual preferences. The issue of 
which factors should be turned over to the user is 
one of the moat interesting and most .important 
topics for exploration. Further, the questicn of how 
the user is to exert this control is also critical. Both 
of these may be dependent, aa with many of the 
other factors discussed, on the type of use. 

The taxonomy presented may not be complete. 
Nor is current usage of these terms always un- 
ambiguous. This paper seta down possibilities with 
the hope that readers will suggest others I have 
trussed. Once a reasonable taxonomy has developed 
we can study through appropriate experiments the 
effects of these different factors in particular situ* 
ations such as student learning. 

For convenience I have grouped the various 
factors in categories. 

/. The time domain 

One of the most critical differences between printed 
material snd material displayed by the computer is 
that the computer can control nmmg. Thus, in a 
sense the computer is more like a film display of 
text or a television display of text rhsn it is tike a 
book. Being able to control timing may enable us 
to emphasize certain material (words, phrises, 
sentences! sections), to show structure, or :o 
increase interactivity, Several timing factors need 
to be considered. 
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/. Overall rate of tat output, In most ousting 
systems text is delivered to ibe screen sc (he fastest 
possible rate, the channel capacity. In communi- 
catioo-based lystems (typically timesharing systems) 
ic is difficult to do much about this; the channel 
capacity or, in computer jargon, baud rate, 
determines the output rate. But stand-alone (single 
user) systems allow us to choose our display rate, 
and to vary the time delay between two characters. 
Thus, the 'image' era appear instantly or at some 
slower rare. The race can be controllable by the user. 
The relationship of this rate to reading speed may 
depend on bow the teat is added to: one complete 
line at a time or one character at a time, for example 
(see section VII). 

Of particular interest is how readability n affected 
by the speed of output and also how actirudinal 
factors depend on this. My speculation would be 
that a large amount of text displayed very rapidly 
would create s negative impression, but this is only 
a conjecture. 

2. Relative rati of tat output. In addition to the 
'generaT slowing of the display rata just discussed, 
special delays might spply to particular parts of s 
document for emphasis. So we could psuse both 
before s key word and after it to give it a particular 
emphasis. The delays might be of different lengths. 
Or we could impose uniform delays after each word. 
A similar approach could help to emphasize phrases 
or even sentences that need special attention within 
material. 

3. End of line delays. If lines are revealed pro- 
gressively, at a fast rate, it may be desirable to have 
delays at the ends of lines to allow for eye movement 
to the next line. 

4. Delays between writing on different pans of tht 
screen. Pauses may help between writing some 
material on one pan of the screen and then writing 
some other materials elsewhere on the screen. 



5. Dtlays bemoan tat and graphics. Although oar 
major concern is with text, it may be (hat the text 
is interwoven with graphic material Perhaps a few 
words will appear, then something graphic will 
happen, then a few other words, then something 
graphic This resembles what goes on at a black- 
board while the instructor H is both talking and 
drawing. In this situation a possible spproach is to 
delay during the transition from text to graphics 
and the transition between graphics snd text, sgsin 
allowing a shift in the viewer's attention. 

//. Word-phrase emphasis 
As already stated in connection with timing, U is 
often desirable to emphasize particular words or 
phrases. A variety of other visual tactics which come 
under the general rubric of types of display are 
possible. These physical methods of emphasis could 
be important iu aiding reading. Fonts, dis cu ss ed 
later, also can provide such emphasis. 

/. Flashing. By flashing wa mean that a particular 
word or a particular phrase appears brighter on the 
screen for a period of time. This might be done 00 
some type of displays by overwriting and other 
types of displays by rapid switching back and forth 
between bold and nonbold type fonta. 

C Blinking. A variant of the above is that a pan 
of the text can bunk off and on. The rate of blinking 
is s variable. < 

X Reverie video. A critical pan of the text might 
be, for *mp h «f«" T in black on white rather than in 
the usual form. 

4. Bold Words or phrases may be in bold. This is 
discussed further later. 

5. Tat movement. Some display systems will allow 
an individual word within a sentence or some other 
larger grammatical structure to move around on the 
screen independent of the rest of the text. Thus, 



the material might oscillate. This text animation 
then would also serve aa a way of emphasizing the 
material. 

6. Color. Another variant in this direction is the use 
of variable colon such as using different colon for 
different grammatical categories of words in a 
sentence. (If color is available, sound may be as well 
Although difficult to incorporate in a textual 
taxonomy, the use of sound can alao, in a variety 
of ways, emphaiuxe words, phrases, or other 
material This might be just the ringing of a bell 
or it could be a much more extensive use. Thus, 
we could use musics! background just aa in a film.) 

7. Underlining. Underlining is s traditional way of 
emphasizing words and phrases in typed material. 
Underlines can flash and blink also. 

& Word-phrase spacing. Material can be emphasized 
by displaying it with more than normal spacing, 
either horizontal or vertical. 

///. Lines of tat 

Io one sense the basic unit of text display on most 
screens is the line. In a 000k the line is the same 
for all users, but wuh a screen 11 can vary wuh users. 

/. Length of tine. Some evidence with regard to 
traditional media suggests that longer lines are 
difficult to read. Thus, an interesting factor to 
investigate is the effect of line length. As with many 
of these items we can let the user pick line lengths. 

I Natural breaks. Often* the positions of 'carriage 
returns' or line-endings are determined by the 
overall consideration of page or screen width. 
Another strategy is to have the line breaks at the 
ends of natural phrases, perhaps combining this 
with s maximum line length. 

J. Hyphenation, Evidence indicates (hat 
hyphenation hurts readability in printed material. 



Nevertheless, hyphenation is vet y widely used in 
electronic media also. 

4. Justification, lines of text can be arranged in a 
number of ways. The most common is the 
rypewriter-uke situation where the lines scan always 
at the same point at the left-hand margin, left justi- 
fication. Often seen in books (but not necessarily 
helpful in reading) is justification at both the left 
and the right ends of the line by insertion of spacing 
in the line. This can be done on disr iay screens too, 
but often the only variable space allowed is that 
associated with the spaces between words; thst is, 
'he spacing between letters cannot be varied. 
Furthermore, even word spacing is controllable 
often at a rather crude level, a full space. 
Justification in convention tily printed material 
involves much more flexibl ' control of ? pace; the 
technique of 'letter spaatg* adda small spaces 
between ail the letters. 

A third possibility is to Justify only the right 
margin, letting the left margin be 'ragged'. The 
fourth possibility is to center line within the 
space allocated for it, so that neiaev the left nor 
the right margin is justified. A vari»ut of this is to 
put random amounts of space at the two ends; we 
might consider this a fifth possibility: random 
justification. 

5. Number of columns. Both one snd two column 
formats are employed in books, sometimes even 
more. These possibilities 1 1 to exist for the screen. 



IV. Space 

A xreen contains both text and blank space. 
Although it ta perhaps straining a point to put it 
into this category, I will include such issues as the 
overall text density 00 the screen in (his discussion 
of space. 
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/. Ttxt density. This might be defined most limply 
as (he rauo of the screen am occupied with text 
to the tocti screen area* Perhaps in nuking this 
computation we should include the space at the 
ends of the Una that would be needed for justi- 
fication, since that is not typically available for other 
use. 

An important item, difficult to quantify, is where 
the blank space occurs. For example, we may have 
blocks of text expressing different ideas; the 
differences can be emphasized if the blocks are 
separated and placed on different parts of the 
screen. The question there is whether the blank 
space can be simply randomly determined perhaps 
by the machine itself, or whether it should be ' 
derermired by s competent graphic designer. One 
might call this the issue of statistical blank space 
versus controlled blank space. Control could be 
thought of as referring either to the placement of 
the material or to the blank space. 

Z Space between Us ten. Normally, letters are not 
written directly next to each other; some amount 
of intervening space is left, usually smalL The space 
between letters is an interesting variable to 
investigate in connection with the present pr oblem s, 
(t might also be described as the size of the letters 
in relation to the space in which they are put, and 
so could be indicated by the same type of ratio 
expression indicated in (1) above. 

X Spaa between lines. Another specific consideration 
is space between lines of text. 

V. Character! 

I. Comparative character site. Utters can themselves 
be of different sue. Many current screens allow only 
a single size letter or only a few static sizes, but the 
possibility of variable size letters is not hard to 
imagine and is already possible with some systems. 
An additional possibility is the mixing of letters or 



various sizes in 'he same presentation, perhaps for 
word-phrase emphasis or for other reasons. 

& Fonts. In print media s variety of fonts will be 
used. Thus, a heading font may differ not only in 
size from the body font, but may be a bold or italic 
variant or s different type face, perhaps for 
emphasis. The font can be a conscious choice, but 
this is rare on today's screens. 
3. Character aspect ratio. The same font of 
characters can be displayed with varying aspect 
ratios, the ratio of a letter length to its- height. 

i Quality of characters. Most computer displays soil 
have very primitive, almost barbaric, characters. 
These characters are often made up of very limited 
dot matrix formats. But some displays, such aa the 
Xerox Star system allow better type fonts. 

VI. Pate aspect 

The traditional printed page was higher than it waa 
wide, with a variety of ratios. CRT screens also 
show s variety of ratios, but except for s few word 
, processing systems the tendency is to make the 
width the larger dimension. Many of these screens 
are TV-based having the aspect ratio of 3 to 4. Some 
systems have s square screen; Although this factor 
may not be important in our consideration, it would 
be desirable to determine that through actual usage 
rather than speculation. 

VU. Overflow handling 

Eventually as text is displayed on the screen, either 
the entire screen or the 'window 1 or viewpoint in 
which the current material is being written will be 
full. The points raised in this section have to do 
with the. action taken under these drcnmiranrrt 

/. Scrolling. Most of the common, inexpensive 
displays use a simple scrolling mrr nanism. That is, 
when the user gets to the bottom of the screen every 
line jumps up by one line, and the new line is 



displayed at the bottom. Thus in this circumstance 
(if the screen is oot cleared) most of the time 
associated with any long text output will be spent 
reading the bottom line, which may appear 
•instantly*, character by character or word by word. 

A useful variant under many circumstances is to 
allow scrolling of more than one line at a time; the 
number of lines scrolled can be chosen either by 
the program designer or by the user. Another 
variant is thst scrolling can either occur 
automatically, or it can occur after the user is 
queried as to whether scrolling should take place. 

Scrolling can be a property of the full screen. But 
we can also scroll individual areas of the screen. 
Thus, some material can remain on the screen while 
other material is scrolled. Several textports (areas 
of the screen) can be manipulated separately. 

An alternative to scrolling is s complete clearing 
of the screen, with text written at the top of the 
screen onze more. Again, this can be done 
optionally with the user requesting it, or it can be 
done automatically when the end of the psge is 
reached, possibly combined with some suitable 
delays-as indicated in the section above. 

A special case of scrolling might involve only a 
single line of text. The material thua is constantly 
being erased and rewritten. 



2. Patimng. Io scrolling the lines always move by 
an integral number of lines, typically one as 
indicated. In panning (or 'smooth scrolling') the text 
troves upward or downward at some steady rate. As 
with other items considered the rate can be chosen 
by the designer, or can be under the control of the 
user. 



3. Crawling. A small amount of overflow can be 
handled by having the entite text move to the left, 
as in some news signs on buildings. 



Vtlt Aids to browsing 

Books are seldom read completely linearly. Some 
books are almost never read in this fashion! The 
typical situation for browsing or searching might 
be where the reader wants to find a particular pirce 
of information- or wants to obtain s general 
impreuion as to what s book is all about. I use 
browsing to include both of these categories, 
^Vpngh they might be split or perhaps better terms 
might be developed. Convenient browsing is a very 
important property. ^ 

/. Indexing. Most books, and much other printed 
material, contain indices. They vary greatly in 
completeness. An index can also indicate the degree 
of importance of a particular reference, the levei 
of treatment, etc Indexes can\be selectively 
displayed on the screen. 

2. Free-form indexing. An alternative' to an index 
is the ability to search for any word\>r phrase, 
possible in s computer environment. V 

3. Pagt flipping. Psge flipping is the facility to be 
able to move either forward or backward rapidly 
in gulps of pages at a time. The rate can be selected 
by the user. In a variant on psge flipping, on each 
page key words or phrases (perhaps user-selected) 
can be highlighted by one of the processes discussed 
above in word-phrase treatment. 

4. Hypertext. Hypertext is a word invented, I 
believe, by Ted Nelson. The notion is to allow a 
type of super browsing capability, where the depth 
of browsing can be controlled from everything to 
a very brief summary of the material to the lull text 
material. An intermediate stage might offer a 
summary of each page or each group of pages, so 
thst one could decide whether that waa the page 
at which the material was likely to be usable. The 
user must be provided with some essy wsy of 
controlling this situation. I should hasten to ssy that 
the word 'hypemxt' also includes other concepts 



besides this, ones that are not directly relevant to 
our present interests. 

IX, Marginalia and crott-rtfertneing 

Notes in the margin, and other similar devices, can 
be very useful. These notes might be 'private', or 
they might be made available to other readers. 

/. Nous. A common procedure with printed text 
is the writing of notes in the margin, a capability 
also related to hypertext. Although this has seldom 
been done with textual material displayed directly 
via computer displays, it represents a possibility. 
Such notes could be later modified or exanded. (see 
also section XL) 

Z Potmen. The user can also enter pointers from 
one passage to another. 

3. User highlighting. A related feature is to allow 
users to impose their own word-phrase control. This 
is similar to underlining or using markers in print 
material 

4. User editing. The user may warn to change the 
text for his or her later purposes. Implementation 
of such a facility will also have to consider privacy 
factors. The user should be able to state whether 
a particular note or pointer is to be readable by other 
users of the same text (if that is possible on the 
system), or if it is to be restricted in some way. 

X. Viewing environment 

Displays are often viewed in non-ideal 
environments. Studies in the area should see how 
environmental factors such as room illumination, 
screen contrast, distance from the screen, etc, affect 
. readers. Factors such as the wearing of glasses may 
be important. 

XL User interaction 

So far we have been considering the usual 'reader', 
a receiver of information. But the computer allows 



the reader to play an active role. In the best case 
we can have full . interaction, as in the beat 
computer-aided learning material. But with little 
work for us beyond just employing existing print 
material on the screen, we an occasionally invite 
the user to type in a summary, with the idea 
understood that such optional input would not be 
analyzed. 

XII. Design of the tenon 

I mention Anally the possibility of special graphic 
treatments set out by a competent graphic designer 
as opposed tu simply using some (random) 
combination of the factors indicated. 

User control 

As we have been noting in our previous discussion, 
one unique aspect of the computer-driven display 
as compared in print medium, is that many of the 
functions which normally are completely deter- 
mined in advance can be rumed over to the user. The 
reader can be provided with many degrees of control 
which are not a* Jlable in print medium. Almost 
all the aspects of screen display discussed so tar can 
be under the control of the reader. 

But with all these new degrees of freedom come 
new uncertainties about bow to employ them. At 
least four interrelated issues need to be discussed— 
the dependence on user characteristics, the question 
of which variables should be rumed over to the user, 
how user control is to be provided, and the issue 
of training users in the effective use of this new type 
of facility. 

Different users may have very different needs 
with regard to the type of control provided. So our 
characteristics cannot be ignored. One would 
expect, for example, that a second grade student 
might best be provided with different types of 
control over the display than that provided for a 
good university student or an industrial trainee. 
Although little research is available as yet, the 



situation might well be very complex with many 
different user properties affecting readability and 
other issues. 

In a given situation it might be entirely too com- 
plicated from the user viewpoint to turn ail of the 
user controllable capability over to the reader. 
Hence, the issue of which variables should be under 
user control is one that requires study. At least two 
issues are important. First, if the user can modify 
the situation in some particular way, toi'l th*. 
majority of the users or any users actually use that 
capability? If something is never used, it should not 
be given to the reader. But a more important issue 
is which user controllable variables will actually 
make a difference to the reader, either in terms of 
the ease of readability of the material or in terms 
of affective issues. 

The question of how the user is to exercise control 
is a rundamental one. How is the user to make the 
decision aa to how to do these things, and what is 
the mechanism by which I** decision is conveyed 
to the computer? These issues are not simple. 

At least three overall strategies are possible, and 
combinations of these could also be followed. 

1. Initial choice. When someone first comes to the 
text or pro gram it is possible to offer the option that 
they could alter the conditions in some way beyond 
the 'default* co: 'ions. This would mean that the 
assumed conditions would need to be told to the 
user, and then the user could alter these if desired. 
Some methods for this initial alteration are possible. 
Cm is a menu-like approach where one points with 
the cursor to a line that is to be altered, and then 
the user is led step by step through the altering 
procedure. 

2. Verbal commands during reading are the neat 
possibilities. It is possible that readen could be 
entering commands indicating change*. Thus, if 
readers wanted the entire program to go faster, they 
might type faster, while the output is appearing on 



the screen. The advantage of being able to use 
English is clear. The difficulty is that the user 
would have to learn something of the vocabulary 
needed for these changes, although the built-in 
vocabulary could be quite extensive and therefore 
flexible. Or this could be allowed only at certain 
specific places in the material. 

3. The third possibility is to allow the user some 
special mode of interrupting what is happening, to 
make changes. Thus, a particular key might be 
reserved for this purpose. When the user presses 
the key, a menu-like situation similar to that 
described in (1.) could then ensue, or a query 
routine could be started. 

These possibilities are not intended to be mutually 
exclusive, but could all be used. Experimental study 
could determine which ones and what combinations 
are more desirable. In any case the careful picking 
of the method of user control could have a profound 
effect on the issue of readability. It may be, too, 
that the user may not want to use any such control, 
but would find it easier to stay with the default 
settings. Again, only carefully done experimental 
work would determine whether this is the case or 
not. 

The final issue is also a critical one— the process 
of training the user to use the facility. This interacts 
with the other factors; a training procedure could 
be rather simple if only a few alternatives are turned 
over to the user. The question of the type of user 
is critical as to what one could provide; less training 
would be needed for the one-time user than for 
someone who is going to be reading targe amounts 
of material at the screen. 

The training process could involve off-line 
material in written, slide-tape, or other media; or 
it could be done directly at the display, using the 
best techniques of modern computer-based learning. 
My general preference would be the second 
possibility. 
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ImpitaMntation comments 

It is not difficult to score text in such a form that ami 
of these various options could be picked randomly 
in experimental situations. Thus* to implement 
word* phrase emphasis and timing and line length 
control, we need only to place markers before each 
word that is to receive special emphasis or perhaps 
different kinds of markers for different types of 
emphasis, and similar markers both before and after 
natural phrases. 

Acano>rifOQtfT«nt»: Uwhjl t u q gm t om rw*9 com* from mtny 
novwluas ineiuoiflg in« roiow*g: S. franklin lUnrwufv of 
Csttvr**, ifwwi. J. rotm iMjnwwr Poryi»cnr«i. J. Harttov 
:Urvv«f»fv of <••♦•». < HOOfMT iUf*vgr*rv of C«ftfOfni«. 5*rtt 
CnuJ. J. Hf«nt lOgiui Eomomtni CoroouMnt. 9. iCuru 
iUn»vor»tv of C«fcrom4. irvmai. M, H\mt tUr*v«f»tv of low*), 
3 J. Pncm lU-wvorfttv of C*Mom*. Syttwnvwao Admmtraoonj. 
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Negative views about computer based learning are 
occasionally encountered, They are held by a number of people, 
often for odd reasons. This paper attempts to respond to certain 
negative views, often seen in the literature, by emphasizing 
positive aspects of the computer as an aid to learning. I will 
not consider all the "philosophical" objections that have been 
made. I will discuss a particular example of computer based 
learning, in the context of earlier efforts, and show how it 
illustrates the ideas. The example, sponsored by the Fund for 
the Improvement of Postsecondary Education, was developed In the 
Educational Technology Center for improving scientific literacy 
in the general populace. 
Some Key Ideas 

1, Active Learning . One of the primary goals of many 
innovative learning projects is that of encouraging active rather 
than passive learning. Creating an active learning situation is 
u prime consideration justifying more widespread use of computer 
based learning. 

I have argued in many papers that the computer, working 
directly with computer based learning modes, furnishes the only 
possibility in today's mass education for making learning a mote 
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active process for millions of learners. -Host other learning 
approaches which are highly Interactive do not work when the 
number of learners grows very large. Indeed, I would even go 
further than that: It will make learning a more active process! 

2. C. A.I, 7 Many negative writers use the term "CAl" for 
computer based learning. Although it Is a matter of terminology 
and so not a critical Issue, I would like to quibble with that 
terminology. I don't think It's very useful, and often leads to 
opposition. First, using sets of initials is abominable and 
doesn't fit in with reasonable English style. Attending a 
meeting In an area with which you are not very conversant is 
usually appalling, listening to people speaking to each other in 
codes. There is no need to do so. The English language is 
adequate. 

Further, "CAP is a "red £lag" for some Individuals. They 
saw, perhaps many years ago, some material with that identifying 
tag, and since then they have felt that everything in which the 
computer is used in an Interactive tutorial mode has the same 
characteristics as whatever they saw. 

The net result is that I do not use the inkials but speak 
rather of computer based learning, Hy emphasis is on learning , 
what the student does, rather than instruction, which is what ' 
someone else does. This emphasizes the notion of active 
learners, already mentioned. 

3. Philosophical versus Practical Decisions . I get very 
worried when people make pedagogical decisions on grounds of 
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! overall philosophy. While It is Important to have an overall 
philosophy in dealing with educational material, we are still at 
the stage where much is to be learned about the nature of 
learning. So we dust be cautious about making decisions on 
statements lacking a good empirical basis without considerable 
practical experience in the area Involved. 

This has particularly been a problem with the use of the 
computer In education. Many people have started with strong 
philosophical positions and have made decisions on how they would 
use the computer based on these principles. One underlying 
principle behind such philosophical points of view is that some 
"right" way exists to use the computer, and that this mode of 
usage can be justified on the basis of argument rather than 
experience. This approach Is fundamentally antl-sclentlfic. 
Empirical evidence Is essential in making such decisions, 
particularly in an area such as learning where no fully adequate 
scientific theories are available. 

I would argue that no "right" way of using the computer 

♦ 

exists. Of the many different ways possible almost all of them 
are effective In some pedagogical situation, The decision as to 
how the computer is to be involved In learning should be made on 
pedagogical issues, often dependent on the particular area rather 
than on general philosophical grounds, 

5, Experience , Very few of the critics of computer based 
learning have had very detailed experience with good, modern, 
contemporary, computer based learning material, I doubt if these 



critics have spent many hours during the past several years 
critically examining such material and making decisions about Its 
use In classroom situations* Thus, I am afraid that the critics 
are often criticizing entire classes of products with which they 
have almost no direct experience, 

6, Quality of Computer Based Learning Material , The 
development of computer based learning material is still In Its 
Infancy, Use of any new learning mode requires detailed 
experience and study of how that mode can best be employed as 
compared with older learning modes, 

Many of the people developing computer based learning 
material are heavily influenced by books and lectures, and so 
they miss just the component of active learning which is so 
important, They tend to transpose books and lectures to the 
screen, and so produce Inferior material. 

This situation Is now changing.. As we gain experience, we 
are producing a wider range of interactive computer material 
which begins to exploit the capabilities of the medium. Quality 
is Improving, He still have much to learn, and even the best 
current material will appear rather crude a tew years from now. ' 

/ 7. Individualization, In addition to the extremely 
desirable goal of creating active learners, another advantage of 
computer baaed learning deserves close attention. The learner, 
being active, Is often replying to questions from the computer, 
The computer analyzes each response, and makes further decisions 
on what learning experiences to present, based on this analysis, 
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as compared to ordinary quizzing is the immediate feedback. This 
, can occur in the obvious fashion of telling the student whether 
the answer is c; is not correct. It can also be much more 
extensive, offering very detailed help sequences cmi exactly on 
the 'type of difficulty the student has shown in the quiz. The 
quiz can, in fact, become the dominant learning device in the 
course as more and more of the learning material is built into 
tl.e quizzes. 

9. Basis of Comparison . One issue that I think must be , 
addressed when one looks at computers in learning is that of the 
standard of comparison. That is, what other curriculum material, 
used in what type of environment, is being compared with computer 
based learning material? Without such a standard of comparison, 
discussion can quickly become unrealistic. Thus, to compare 
computer baaeJ learning, material, or any proposed new mass 
educational approach, to what goes on when an extremely skilled 
teacher works with two or three students is unfair and 
unrealistic. It is more than unfair-given the current problems 
of American education, such an approach is highly misleading, 

To state this in a different fashion, I would not claim that 
the computer, used in any way in education, will be superior to 
the extremely good instructor working with only a few students. 
This very small group learning situation, with excellent ■ 
teachers, is extremely rare In education at any level. It does 
occur, for example. In some British universities. Almost every 
• important physicist in England in the second half of the 
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These decisions then can reflect closely the needs of the 
Individual learner,. and so the, material can be different for each 
person. We can thus achieve a degree of individualization 
possible with no other media in environments with large numbers 
of students. 

Furthermore, the time to go through the material can be very 
different. This is to be contrasted to the typical lecture mode, 
still the primary mode of teachin g (not necessarily learning) 
almost all courses in our universities and our school system, 

8. Quizzes Via Computer . The computer can be used to 
deliver, grade, and record the. results of quizzes. This is one 
of the most powerful modes of computer based learning. 
Unfortunately it is often used poorly. 

Multiple choice quizzes are an abomination and should never 
be used in learning situations, Or, perhaps to be a little less 
radical, multiple choice should be used only rarely in situations 
where Its use simply allows some change of pace from other types 
of quizzing, In the extensive collection of on-line quizzes 
developed for our beginning physics course at Irvine we use 
multiple choice extremely rarely, There is a belief, usually not 
verbalized, that multiple choice has something to do with 
computers. 11 did, as long as one was dealing with mark sense 
cards and other stone age computer technology. But multiple 
choice questions need never appear in good computer based 
quizzing situations. 

The most effective aspect of using computer based quizzing 
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nineteenth century worked with the same tutor. The tutor was 
unknown as a scientist, but nevertheless the evidence Is clear 
that he was an extremely competent teacher. 

5 I claim that the computer, used In computer based learning 
modes, could lead (I emphasize could rather than will) to an 
order of magnitude Improvement In what happens In our typical 
class situation, whether we are looking at the K through 12 
system, the university system, or adult education. 

Let us take the beginning physics course in unJ.araitiea as 
an example. What is the typical situation? First, relatively 
few books dominate the market, beginning with Haliiday and 
Resnlck. Furthermore, these few books are extremely similar in 
both content and problems. In addition to the books a second 1 
major learning mode provided in most beginning physics courses la 
the lecture, typically closely parallel to the text. The book 
and lecture do not satisfy In any way the criteria of active and 
individualized learning. The class Is often large, but even when 
the number of students is relatively small, a lecture cannot 
provide an active learning environment. A few charismatic 
lecturers may furnish strong motivational impetus, serving as a 
role model, but that Is very unusual, not the common situation. 

Many of these courses also require homework, graded, 
typically rather unevenly, by hordes of advanced students. 
Feedback is delayed: The time between the moment the person 
completes a homework problem and the moment when this student 
gets information from graders, 1b likely to be days and sometimes 




longer than a week. Because many people are Involved In grading 
and' because there are varying capabilities, the feedback is of 
highly varying quality, Often the emphasis Is simply on 
assigning a grader with little lieip" ptuvliW, 

i 

Instructors will have office hours) but 1 E wejdivlde office 

hours by the number of students In the course, we detain an 

extremely small number in the majority of situation* So again 

little .Individual attention is available. 

I don't want to sound overly pessimistic taut the* 
■ 

physics course. But when critics say that computers can oiUy 
Jeach problem solving the way the developer of the materials 
wants the problem solved (whi&h may or may not be true) , we 
should look at what happen^ with learning problem solving in the 
typical beginning physics course. In looking at computer based 
learning wt must look at the existing classroom situations and at 
practical ways to Improve existing situations to get a reasonable 
standard of comparison. , 

Alternatives to typical beginning courses exist. Self- 
paced, mastery systems, usually called Keller plan or PSI, 
• provide an alternative. Typically these have been successful 
only with relatively small groups. The Feller plan articles from 
MIT are very revealing about their difficulties with large 

physics courses. In developing such a variable-paced course, to 

\ 

overcome the problems of the typical course, we were led to : 
develop our system of qn-line quizzes In the self-paced, mastery 
tradition. 
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The situation with regard to physics is not unique. I could 
have discussed almost any subject area* I picked physics because 
it Is the one most familiar to me, and because I have developed 
extensive material in this area. 

10, The Commercial Have . It is my position that computer 
based learning material, because o£ interaction and 
individualization, can be extremely effective, particularly as 
compared to any other currently likely possibilities which might 
lead tb better educational systems for large numbers of people. 

• In another sense, particularly from a long-range point of 
view, it d ocs not matter in the slightest whether computer based 
learning material is or isn't better, In either case, we will 
see more and more of it until it will eventually become the 
dominant educational delivery system! 

The question of whether or not compute based learning 
material will become widely used is not one that will be decided 
on rational. grounds, based on the effectiveness of the learning 
situations provided. It is not an "academic" issue. Rather, 
current great difficulties with education, plus very powerful 
marketing forces, will be the major factors/ These factors will 
* dominate, and the rational Issues of effectiveness of this type 
oC learning will not be major considerations. The only question 
is whether the future of the computer in education will be 
desirable or undesirable; that issue is still unresolved. 

We can see clear signs of rising commercial interest, 
although 1 do not have time to loot, at the issue fully, Almost 

- 9 - 

\ 

\ 




every major textbook publisher, plus many new companion, are 
currently gearing up to market computer baaed learning material. 
Many computer manufacturers are also moving in this direction. 
The amounts that these companies are investing are growing 
rapidly. 

He need only look at the textbook market to understand why 
publishers are moving toward computer based learning, even though 
there is great uncertainty about the marketing data and marketing 
otrategles. Textbooks are not a growing market, so these 
companies eagerly seek other narkets, while attemping to protect 
their current market. 

1 would not want to claim that all the material from these 

companies Is first rate or even reasonable. At the present time 

much of It is poor. That even poor material can be marketed by 

major companies is an Indication of the need for this material in 

the schools and universities. The list of existing companies 

involved is Impressive. It includes such companies as Harcourt, 

Brace, Jovanovich, Harper S Row, John Wiley, McGraw-Hill, Random 

9 

House, Scott Foresman, Science Research Associates, and many 
others, The critical point is that the important decisions will 
increasingly be made by commercial firmn, not educators . 
An Example - Elementary Science Material 

So far the discusrion has ten theoretical, talking about 
computer based learning in abstract terms, This worries me for a 
variety of reasons. We can understand the issues only if we 
examine actual examples. ,In this section I consider an example. 
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Three aspects will be considered: 1) An important pedagogical 
problem; 2) attempts to solve that problem (which In terms of 
mass education did not succeed), and 3) a computer based learning 
variant of these attempts which holds promise of succeeding not 
only within schools but also in homes. 

It would be ideal if I coulcl supply readers access to the 
computer based learning programs ;l will describe, It Is 
difficult to convey the notion of a highly interactive graphic 
learning experience with the passive print medium. But I will do 
the best I can, hoping that readers can later try the modules, 

The educational problem to be considered is important, not 
only In the United States but all over the world, Present 
science curricula, particularly as reflected in what the average 
student learns , do not bring students to an appreciation of the 
nature of scientific knowledge, Hose approaches are vocabulary 
oriented, presenting words and "facts/ These courses convey 
little understanding of how the scientist goes about arriving at 
such vocabulary and information. Science is almost presented as 
a new type of religion; the conclusions must be believed because 
the scientists say sol Very few students come through the 
educational system in the United States with any understanding of 
the nature of science, Yet," in a society heavily influenced by 
science and technology, such understanding is Important, H 
similar situation exists elsewhere. 

The realization that a difficult problem exists and that w 
are failing to solve it Is not recent, Major individuals In 



science education have been concerned with scientific literacy 
for many years, Several thoughtful projects concentrated on this 
problem In the I960 1 a. Among these projects were three 
elementary curriculum efforts that developed materials closely 
related to the ones to be described, the Elementary Science Study 
project (ESS), centered at the Massachusetts Institute of 
Technology, the Science Curriculum Improvement Study (SCI5), in 
Its initial form developed at the University of California, 
Berkeley, and the AAPT material, A Process Approach, The SCIS 
project was published in two forms and marketed by two 
publishers. High school curriculum developers also were 
conscious of the problem of scientific literacy, as seen in such 
courses as Project Physics, 

In these curricula developments Individuals distinguished by 
their knowledge cf science and their pedagogical backgrounds 
contributed, The SCIS project was Important for Its pedagogical 
contributions, independent of the unity produced; It Introduced 
the Important developmental psychology notions of Piaget to many 
teachers and researchers in the United States. 

The materials produced In these projects were Impressive to 
the professionals, They did focus on the -problem of bringing 
students to understand how the scientist works, how scientific 
theories were discovered, They used a variety of subject areas; 
the materials were thoughtfully developed, tested, and improved, 

Let me mention one example, the batteries and bulbs unit, 
Students are given some batteries, some light bulbs, and some 
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wires. As far as pc:slble, the batteries are all similar as are 
the light bulbs. Students are told that their initial task is to 
make the bulb light. They experiment with the equipment with 
this task in mind, Students are not told how to light the bulb) 
rather they should have the actual experience of a small 
empirical discovery, In some variations additional help might be 
given to the student who was slow in finding how to light the 
bulb. 

Lighting the bulb is only the first task. After the student 
has succeeded in doing this, perhaps in several different ways, 
the next stage is to verbalize what is necessary to light the 
bulb, arriving at the notion of a continuous circuit which uses 
both terminals Ot the battery and both terminals of the light. 

By a series of additional empirical Investigations , the 
students gradually develop a theory or model of what is happening 
in the circuit. In the course of this investigation such terms 
as "current" and "resistance" are Introduced, although not in the 
full-scale sense of circuit theory. It is the' development of 
this theory and model, from the empirical basis, that is the 
focus of the activity. I emphasize that although the student 
learns something about batteries, bulbs, and electrical circuits 
in the process, the major focus is on scientific methodology. 1 

What happened when the SCIS, ESS and Process Approach 
materials were introduced into classrooms all over thy country? 
The results were far less than many of the developers 
anticipated. Indeed, these units are little used In American 



schools. These projects were disasters in schools, 

Furthermore, examining classes in which the materials are 
used often reveals little resemblance to what the developers had 
in mind. The difference betwpen on ideal curriculum, often 
developed with rosy notions about classes, and the curriculum 
material in % average class, with the average teacher, is often 
tremendous. In thi» s ^ nntan f Ms proved to be the case. The 
new materials demanded the active participation of each student. 
They differed very much from the conventional class situation in 
which the teacher was often the only active individual. 

Any Innovative curriculum development has a major problem 
associated with preparing teachers to handle the new curriculum. 
This problem had to be faced to make the new elementary science 
materials usable in every American classroom. Unfortunately, the 
standard mode is quite inadequate; what happens, almost 
inevitably, is that courses for teachers ?re offered cither after 
school or In the summer at local community colleges and 
universities. These quick in-service courses have long been 
looked upon as the best way to handle the problem. Even at this 
time Congress is voting money for this purpose. During the 
period of the 60's and early 70 1 s the National Science Foundation 
s P ent; ten times as much money on these teacher training 
activities as they did on the curriculum developments themselves! 
Yet this money was, as judged at the time, entirely Inadequate 
for the problem at hand. Bringing classroom teachers in for a 
Csw weeks of Intensive work was entirely inadequate to change a 
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lifetime of habits and beliefs. For most teachers no such course 
was available. Even pre-service courses in many teacher tracing 
Institutions were relatively unaffected by the new curriculum 
materials. 

The teacher training problem in connection with bringing 
students to understand scientific methodciogy was Indeed 
difficult. The science background of elementary teachers la very 
weak. Often the college preparation of elementary teachers vas a 
watered-down physical science survey course, designed 
particularly for teachers. These courses in almost all cases had 
little of the feel of what science Is all about and were often 
simply crammed with "facts." So very few teachers had adequate 
preparation, 

Even a teacher who understood the nature of scientific 
activity often had difficulties in the classroom. The problem 
was one of time and numbers. Students needed to work 
Individually and to receive Individualized aid and attention. In 
the typical classroom the teacher has too many students and too 
little time to allow extensive Individualized attention for each 
student. Only a very small amount of Individualized attention in 
all subject areas is possible. 

Another unexpected practical problem occurred In classes. 
The units depended on kits of supplies (batteries, bulbs, wires 
and other things). When the "second" use of material occurred, 
components were missing or equipment needed attention. Batteries 
wear out, light bulbs burn out, things get lost. Keeping the 
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kits working was a sizable logistic chore; a listing of the 
"batteries and bulbs" supplies occupied a full page. Teachers 
became disenchanted with kits, and even schools that had the 
materials found them unused by the teachers. Again, there were 
exceptions, but this was the typical situation. 

Even if the materials had worked perfectly In classrooms all 
over the country, within a short time after their development the 
problem oE other people in our society still exists. If one 
starts at the elementary school, It takes a period of forty to 
sixty years before "everyone" has the new concepts. The problem 
of scientific literacy Is not only a problem of elementary school 
but Is also a problem throughout our educational system, 
including adult education. 

The net result of millions of dollars for curriculum 
development and in teacher training was disappointing. The 
problem of 'science literacy remained. The excellent curriculum 
Ideas could not be applied to the existing situation In schools. 
This Is not to say that the materials did not work In many cases. 
Indeed, when a teacher understood the Intent and was wlHlng to 
spend the class time, the materials were very effective. But 
their total Impact on education In the United States has been 
very small. 

Science Literacy via the Computer 

Our group at Irvine became interested in this problem 
several years ago. It seemed an excellent opportunity to use the 
computer for a major Instructional problem that so far has been 
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Intractable using conventional pedagogical strategies. Computer 
development in this area need not start from ground zero, but 
could be based on what had already been learned in the existing 
projects. The advantages of the computer, providing an 
interactive experience [or each user and individualizing the 
experience to each user, were missing in the typical American 
classroom when the materials were used in their conventional 
mode. 

For several years we were unsuccessful in finding support. 
Finally in August 1979 the Fund for the Improvement of 
Postsecondary Education awarded $200,000 to develop computer 
based scientific literacy ma utlal. We decided, given the 
emphasis of the Fund, that the major testing environment should 
be the public library . 

The choice of the public library as our major arent for 
formative evaluation was carefully considered. We wanted our 
units to work for all students, whether those students had any 
previous experience with computers or not. We were also 
interested in future widespread use with home personal computers. 

We were very concerned with motivational issues. The 
library funrlshes a good place to determine where the programs 
"lose" people, 'lose their attention. When tested in classroom 
environments, the students are obliged to continue even if {he 
unit no longer excites them. No such pressure exists in a 
library; a student who loses Interest will get up and leavel So 
simple data gathering could show us where the programs were 



motivationaliy weak. 

The modules developed in this project have been well 
described in other literature, so no attempt will be made to 
desciibe them fully. As of this writing, they are in final 
development stage, modifications performed after the testing. 
Approximately fourteen hours of material concerning scientific 
literacy are available, assuming an average student user at a 
display, However, our experience in the public library testing 
Indicates that a given user would often run a program many times, 
coming back day after day. 

One of the units concerned batteries and bulbs, as already 
described. He were fortunate to work with several people who had 
great experience in using this material with students at a 
variety of levels. Among those involved in developing the 
materials were Arnold Arons (University of Washington) and 
Francis Collea (California State University, Fullerton). So we 
were able to benefit from the experience of the earlier 
curriculum development projects and from the experience of 
teachers who had used the material extensively, 

I have already suggested that the best possibility for 
batteries and bulbs is for readers to see the material on the 
computer. For many this is not currently possible. Hence, I 
describe th6 unit briefly. 

Each of the eight modules occupies about fifteen minutes of 
an average user's time. The user may move between these modules, 
and sometimes the program itself will suggest or enforce such 



movement, The material is highly Interactive, A group of users 
estimated that fifteen seconds elapse between typical student 
interactions. So this is in no sense a lecture or textbook 
environment, but rather an interactive environment, As far as 
possible students are asked to come up with the important ideas, 
If necessary, they are given partial help so they can still make 
part of the discovery. 

We decided to simulate batteries, bulbs, and wires on the 
•computer. It is not yet clear that this is a wise decision. In 
another module we do require equipment at the display, But, given 
the difficulty with kits and our desire to have these materials 
run in schools, libraries, other- public places, and homes, it 
seemed dosirable to simulate equipment. He hope that we and 
others will compare experimentally the advantages and 
disadvantages of using equipment. 

The first module, as with the original unit, has as a goal 
the lighting of the bulb. The battery and bulb are displayed. 
The user points to places to attach wires, and they are drawn by 
the computer. At first the only feedback the student receives is 
that the bulb lights or not, except that we do point out short 
circuits. The program keeps an internal record of how the user 
has been proceeding. If success is not attained in a reasonable 
time, very specific aid Is given by a 'tutor' related to the 
student performance thus far, As with other parts of this 
program, eventually success is assured for almost every user, 
although It takes a longer period of time for some than for 
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others. We follow mastery learning ideas, assuming that everyone 
will light the bulb through empirical efforts. 

The b<.cH module is a competency based test to see that 
students who light the bulb refcUy understand what must be done. 
If necessary, as demonstrated by the tests, other aid on the 
formation of simple circuits is given to the student. Students 
do not view this as testing plus remediation as perhaps an 
outsider might. Rather, the student sees it as simply part of 
the process of learning with the computer dialog. 

In the third module we start with another empirical 
investigation, placing various objects within the circuit to see 
whether or not\he bulib lights in each case. After some data is 
available, the mode switches and students are asked to make- 
predictions about whether the bulb will or will not light. The 
emphasis on predictions is very strong in the scientific literacy 
modules, Thus, in the last part of this program many predictions 
are based on the model of the circuit behavior introduced. 

Other modules proceed to develop the model, again based on 
empirical information. We avoid vocabulary until there is some 
reason necessitating the use of a term. The terms "current" and 
••resistance" are introduced In this fashion. 

Will batteries and bulbs and other modules we have developed 
concerning issues of scientific literacy solve the major 
educational problem raised? Does the material do what it is 
supposed to do? Unfortunately we do not know. We are convinced 
it will work with very general audiences, based on our library 
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rial, conducted by skilled evaluators, indicates that at 



Our information interviews of students who have used 



Success clearly will depend on their wide availability. 
We believe that these materials arp ver-y promising, 



many cases students do succeed. In almost all cases the 



considering the problem we began with and the previous 



are happy with the material. But no full-scale 



experiences with that problem. I do not see any other approach 



)n has taken place. 



to the problem which has the potential of being so videly useful 



: is typical. 



following conversation between a student and an 



as this one. I regard this case as typical of many of our major 
educational problems. 



»w at the beginning you were just doing some experiments 
?, trying to light the bulb), and now at the end here, 
are answering all of these things right. I think you 
Lly learned pomething. 

i fun. It's better than in science, because in science 

rere doing this stuff and it was' boring. 

: were you doing In science that was so boring? 

we were doing positive and negative charges and stuff 
i that. And it really wasn't fun. But this teaches you 
r, it lets you try again and again. And it's not going 
irade you or anything; it just/give3 you a lot of 
ices. I think it's better. 

i more extensive testing would be needed to determine 
n students' knowledge about the nature of reienct. 
:her factor must be considered. These materials, no 
>w good, will not solve the original instructional 
inless they are very widely distributed. AL the present 
: is not happening. We are currently talking with 
distributors about the marketing of these materials. 
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i pleasure to return to Holland to both renew old 
•s and to participate In the opening of your new 
:he computer In learning. Centers of thin type are 
f Important In the world and Holland la to be 
>r moving In this direction. 

ipurer will become the dominant' way of leatnlng In our 

a relatively brief period of tine. That la, more 
ie near future will learn Mora things from compjters 
ty other learning modes, Including books and lectures, 
ie beginning of serious development of computer based 
erlal. In thla paper I look at some of the key 
lved In this massive change In our educational 

School s 

let me note that computers ate appearing v#ry rapidly 
11 over the world. The moat reliable figures from 
tatea, from the National Center for Educational 
ithln the Department of Education, suggest that at 
e are approximately 150,000 personal computers In 
he United States, about an average of two per school. 
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The rate of purchase In approximately 100,000 per year, so this 
number *U1 be Increasing rapidly, In recent years, roughly 
speaking, the number of computers In schools In the United States 
has doubled each year. There Is nothing remarkable, however, 
about the United states data. 1 understand that In England at 
the present time, for example, there are between two and three 
computers per school. While l n the United States many schools 
still have no computer, perhaps half the schools, in England it 
Is the government policy that every school should have at least 
oite . 

What happens when these computers appear In th© schools ln 
not necessarily favorable unfortunately. Two major problemn, 
problems which will be Important In much of what I soy here, 
affect this, rirst, extremely little good curriculum material 
employing the computer Is available^ This Is true whether one Is 
looking at the teaching of programming, the teaching of broader 
aspects of informatics, or whether it is using the computer and 
nome formal computer based learning material to learn some 
subject area not necessarily concerned with computers. The lack 
at first rate curriculum material Involving the computer Is a 
tremendous deterrent. Although much Is now commercially 
available within the United States, this material can only lie 
charncterlxed overall as poor and often trivial. 

T hr second major problem to^be considered Is related to 
this. Not only Is little decent learning material uslnq 
computers of any kind available, almost all the teachers 
currently in schools and all over the world have little 
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nee with compute*^, particularly little understanding of 
computer U used in\learnlng situations. A teacher whose 
mputer background was curing a brief workshop or aome 
eriod of time is poorly prepared to teach programming 
they have likely picked up a style of programming that 

0 resemblance to good modern programming, and they often 
anguage that should not be used. 

r Advantages 

have argued (in Learning with Computers , Digital Press) 
e computer is a very powerful aid to learning in many 
Although I do not intend to make a full case, it is 
nt to mention the main advantages of the computer in 
^n. 

regard the most Important advantage as interact Ion . The 
c can provide frequent and highly relevant feedback, based 
tly what the student han been doing. Questions can be 
nd responses can be carefully analyzed, all within the 
everyday language of the student. As numbers have 
:d In our educational systems, we have been less and less 
afford a highly compentent tutcr who works with one or 
three or four students. Rather, we have moved toward 
)ductlon methods, where students play a passive role in 
' n 1 nrj process. 

r the first time by using the computer we have the 

1 Ity of providing an Interactive education for every one, 
: the children of the very wealthy. This Issue is 
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extremely Important, not only in the developed countries of the 
world but in underdeveloped countries which need to make rapid 
Improvements In their educational systems. 

The two key issues in interaction are how frequently the 
student does something and the quality of each of these student- 
learning interactions. Not all interactions are equally useful 
in learning. Thus in our activities in the Educational 
Technology Center we avoid almost entirely the use of multiple 
choice, even in quizzing situations. He consider this to be an 
unpleasant form of interaction, one that has njany negative 
aspects in learning situations. 

One consequence of interaction is the possibility of 
individualizing the material. Most learning material delivered 
today in books, the typical classroom situations, treat everyone 
the same, But one person may want to move through the material ' 
at a different rate than another person, or one person grasps the 
material quickly but another ne*da other pedagogical strategies 
to understand what is happening besides the one used initially. 
Some students have weaknesses in background. Thus a student in a 
science class may not be able to use, in a functional sense, some 
critical mathematical technique. Lip service is paid in 
education to the notion that different students Lam in 
different ways, but unfortunately our current learning structures 
do not permit us to react reasonably to this. The computer, 
because It Is interactive and therefore pays attention to what 
the student is doing, can with well developed material 
Individualize the learning experience. 

- 4 - 



9 

ERIC 



feedback. All the record keeping and much detailed aid to the 
student, aid which Is highly relevant to an error that the 
student has just made, Is available. Students agree almost 
unanimously that the qulizes are learning material. Testing 
becomes a learning mode. In a sense, the tests are the coursei 
in that they are the major learning facility Cor almost all the 
students. This way of organising courses Is possible only with 
the computer/ It has considerable promise. It resembles the 
situation of pre-testlng and post-testing, except that the two 
forms of tests plus the learning all take place with the computer 
based tor mat. 2 

The development of this material was funded by the National 
Science foundation. 

The second set of computer based learning modules concerned 
scientific literacy, the nature of science. It Is aimed at a 
broad audience, from young children through adults. The material 
has been widely tested in schools from junior high school through 
universities, and within public libraries and science museums. 

When the science literacy units are used In the library no 
one Is there to "help" the student, with either the computer 
Stalls or with the subject details. Thus the materials are 
completely self running, and therefore also suitable for the home 
environment. The dominant pedagogical approach for the literacy 
materials was suggested by Robert Karplus a numbor of years ago, 
the learning cycle. Karplus derived these Ideas from his study 
of Plaget In connection with the Science Curriculum Improvement 
Study materials. The learning cycle suggests three stages In the 
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I process. The first is experiential! the students have 
ices which lead to better Insight and which prepare the 
the more formal learning activities of the second phase. 
:hird phase students demonstrate what they know through 
or some oth'.r way and receive additional help it 
y. The student should also be able to apply the Ideas to 
eas. Karplus sees these three features aB essential to 
I* 

»re are six units available, each taking a student 
one to two hours to complete on the average. Bach of t!ie 
has a subject area In science and mathematics, but, as 
id, the emphasis is not on the subject matter Itself. The 
e broken Into fifteen minute modules, so the students can 
>und within the materials. The project was funded by the 
: the Improvement of Postsecondary Education within the 
snt of Education In the United States. 
; Educational Technology Center develops many types of 
Ls, Buch as the two Indicated here. We also are Involved 
srch In using the computer more effectively in education, 
ilso consult with schools, universities, and companies on 
: based learning. Some of this consulting involves the 
33 to train people to prepare such material. 
Ion of Comp uter^ Jjaged_Learn lnq Ma terials 
ccy factor for the future will be a system for producing 
jh quality computer based learning material at reasonable 
Imatable costs. One of the difficulties with so much of 
irlal available at present Is that material ls produced by 
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cottage industry procedures, and therefore often does not reflect 
the standards necessary for good learning material. 

The major point to emphasize is that developing good 
computer based learning material Is fundamentally no different 
from developing good curriculum material of a ny type . Thus to 
make an educational film or to write adequate text material Is a 
long, difficult and expensive process. It is foolish to believe 
that there are short cuts to develop computer based learning 
material which would make It ^jch cheaper that other competently 
done material. Many of the details of producing adequate 
computer learning material are similar to the details associated 
with other curriculum developments. Thus the pedagogical 
development stages and the various evaluation procedures are 
similar, no matter what the medium Is, although we will point out 
some differences. 

In understanding the ^ion of computers in education, 1 
find it illuminating to consider the history of textbooks. 
Textbooks revolutionized education, changing the major learning 
mode In almost all areas. Like our current interest with the 
computer, the textbook was baaed on a technological innovation, 
che invention of printing. In the early days of the printing 
press If a teacher wanted to use one of these new fangled things 
called a textbook, the only possibility was to buy a printing 
press and a set of type (undoubtedly the teacher was told that 
there was an easy way to use the printing prcaBl) and to proceed 
to learn the art of printing. The teacher was probably not very 
good at this and only produced o few rather mangy books for use 
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the teacher' 8 own daises. Eventually a system 
d an Industry developed around that system which 
necessity (or teachera to use the printing press* 
hot of a book Is usually concerned just with the 
details as reflected In the preparation of a 

A very similar situation exists with the computet, 
ose parallels. 

ike a few brief comments about the stsges Involved In 
xk) computer learning units. 
leal Design J 

»rtant aspects nust be considered In connection with 
Jeslgn. The Initial planning session may Involve 
ill over the country, engaging In a brainstorming 
;y creat) a specification document for each of the 
i developed. It le Important that research on 
teaching In the subject area be examined. For good 
iterlal we want to employ the best people In the 
y. This design specification can be sent out to 
llvlduals who did not draft It for review and for 
iprovement. 

the detailed pedagogical specification of eacii unit, 
do this In groups of three to five Individuals. One 
t In the group will be highly competent teachers who 
i experience not only In teaching theory but also In 
ndlvldual atudent problems. One other person In the 
eflect the research background, and another person 
inly experienced In producing computer baaed learning 
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material. In the case of the Educational Technology Center this 
last person Is usually a staff member of the center? other people 
involved may typically come from outside our group from other 
locations. 

He find It desirable to do the design work In Intennlve one 
or two week sessions In pleasant Isolated locations, avoiding 
constant Interruption from telephones or other people. He want 
our designers to concentrate Intensely on the activity. 1 will 
not discuss the strstegles we use for recording the results of 
the design group. The specification of the units can be given to 
the screen designers and coders In the next stages In the 
process. This document csn be reviewed as with the preliminary 
design document. That la, It can be sent to others with 
Instructions on how to read ltj changes can be made based on the 
suggestions received. 

Some training of the detailed design group Is needed, 
usually the first morning. The critical aspect la to understand 
the nature of the computer as a learning medium, mostly by 
looking at many examples of good and bad practice. We do not 
believe it Is valuable to teach these people anything about 
computer languages whatsoever; the lsBues of the technical 
details associated with computers and the Issues of how to use 
the computer effectively In a pedagogical sense ate unrelated. 

Nothing has been coded In this stage. One of the keys, we 
believe, to effective material Is to separate pedagogical Issues 
from technical or coding Issues. The Issue of how It Is to be 
programmed should not Influence the pedagogical design, the 
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factor in producing high quality material. — 
s en Design 

important aspect of computer units 1b how they are 

ed on the screen, both in space and time. The 

ors, in some design groups, may make detailed design 

ans, bjit these suggestions usually do not give a complete 

ition of the screen. Instructors, ipdeed, may not be the 

3ple to do It. Ideally, a good graphic designer, 

i to educational issues, should be involved, working with 

i 

Ive software particularly prepared for such individuals. 
2n, however, in, our projects at the Educational 
y Center, we have not been able to afford such a graphic 
In such a case, a series of general principles will be 
such as the following: 

Use blank space freely— it is free on .omputer systems. 

Remove any information not needed— the screen should 
history of what has happened in the last fiv minutes. 

Use as much visual information as possible to aid the 
teak on understanding verbal information. 

Use timing and other devices to stress key words or key 

Thus a key phrase might blink. 

Use timing to aid readability. Stop when you have 
>r question marks. 

Use short lines to avoid reading errors. 

Koep the natural phrases together on the lines, if 

rather than splitting them across several lines. 

Use a variety of textual styles, such as the occasional 
- 11 - 
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use of a text which is right justified but not left justified. 

Screen design furnishes another stage where review and 
evaluation can be important. That is, the screen designs, 
created on-line by the designer, can be reviewed directly at the 
Bcreen by other designers,- and they can suggest changes to be be 
made. 

3. Writing the Code 

The next stage In the development process is to write the 
necessary programs to carry out the logic Indicated In the. 
Bcrlpts produced In pedagogical design. If there has been a 
separate screen designer, it will be not necessary to enter text 
material as that will all be present. Simple graphics will also ~ 
be done, but the coder may need to do complex graphics such as 
that involving animation. 

The basic principle In coding is that, as these are large 
complex programs which will need to be modified frequently (in 
the formative evaluation stage to be described later) , and 
possibly transferred f :o a variety of machines, it is necessary to 
follow the best practices of modern software engineering. I do 
not review those practices in detail, since they are amply 
described in the programming literature. Unfortunately, most 
computer based learning material developers seem to be completely 
unaware of these practices and so a number of unfortunate 
directions have become common. 

Programs should be v ritten in such a fashion that they are 
as readable as possible. They should be written in a. highly 
modular fashion. They should be amply commented. Variable names 

- 12 - 
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Augment Pascal with a whole set of procedure*) for this purpose. 
<• Review Eva luation of the Running Program 

At this stage we now have the first running program. Two 
types of review and evaluation activities are critical at this 
stage, the Internal review of the material and an external 
formative evaluation. 

In the Internal review the material will be* run over and 
over by staff members of the project. They attewpt to find out 
where the program does not wor^ by trying many alternative 
branches. They will also see where the pedagogical Ideas which 
seem good In the discussion group and on paper do not seem to be, 
adequate In an actual running program, emphasis on Improving the 
Interaction Is important at this stage. Usually the program will 
go through many versions because of Internal testing before it Is 
available for full-scale testing with the target audience. 

The formative review process is the first tine that the 
units are used by the people It Is Intended for. The formative 
evaluation Is perhaps the most critical of all the stages, except 
for the 'etallftd pedagogical design* In assuring the eventual 
effectiveness of the material. One of the great advantages of 
computer based learning material Is that It «s Btlll possible to 
make changes at this point because computer programs can be 
easily changed If they are wen written. Kith books, for 
example! change Is almost Impossible after the book Is printed. 

The computer can collect much of the Information In t|ils 
Initial testing with typical students. The student responses can 
be stoud selectively, and sc we can find where the program Is 

- 14 - 
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ponaive to the student language. The students can be 

to make comments. We c*n keep date on different paths 
hrough the material. We can do testing on-line to 
ne what the student has learned. 

her evaluation procedures are also useful. A skilled 

or can Interview students or can observe students 1 

r. Detailed studies can be conducted using videotapes of 

o using the materials. ' 

veral cycles of formative evaluation are useful. A& 

ing numbers of students are used in ench cycle, the 

iono may becote quite expensive, so budgetary 

lotions may restrict how much may be done at this stage. 

e final summative evaluation is a more difficult issue. 

initiative cvalutaions are expensive. They should be done 

the initial group by competent evaluators and with 
e funding, so that sizable numbers can be involved in 
ests. They should also not be performed until wdll after 
er-lals are developed to allow the maturing of the product. 
n_q_ La nguages and Systems — A False Dir e ction v 
fore lef 'Ing the issueb of producing computer based 
q material, I wish to contrast a aystera of the type just 
ed with another commonly used system which, I would argue, 
n one of the main deterrents to producing effective 
r based learning material thus fat, the use of specially 
d authoring languages or authoring systems particularly 
h materials. 

ndreds of millions of dollars, and perhaps even billions 
- 15 - 



8j 



\ 

of dollars, have been spent on developing such oysteiio. With few 
exceptions, languages with large financial — ^rt (such as 
Tutor) have been used in writing materials, mat alone should 
indicate that something is w;ong with the approach. 

The initial philosophy beyond many of these languages is a 
"Renaissance man" philosophy. Instead of the caieful separation 
of tasks by using people ok! different abilities necessary for the 
type of production system envisioned in the last section, one 
person does it all. It is as if we were back to owning one's own 
printing press and preparing booksl Developing curriculum 
water ial is a complex activity and cannot reasonably be expected 
to be carried out by a single individual, regardless of the 
medium envolved. Inevitably the pedagogical issues become, in 
authoring languages, confused with the necessities of programming 
and the primary decisions are not made on pedagogical grounds. 

Although Many of authoring languages and systems start with 
this single person ^philosophy, some recognize its limitations and 
begin to use a strategy more like the one reflected in the last 
section. Then the author no longer uses the system, but separate 
individuals are involved. Then the objections to the authoring 
languages and systems become quite different. They are then 
programming languages and they can be evaluated as programming 
languages, the same modern criteria we *ould use for evaluating 
any other programming language. The authoring languages and 
systems are mainly very primitive as compared with modern ' 
programming standards. They do not take into account what is 
known in modern software engineering. 

- 16 - 
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ler, for example, Tutor. Tutor It an old creaky 
latlng from about fifteen years ago. Its structure Is 
r that of an assembly language macro language, with a 
I, a command field, and an argument field. It has a 

of very complicated commands, over two hundred. No 
I language designer today would ever develop Btich a 
Yet Tutor has been the most widely used of all such 
anguages and probably also the most widely Imitated, 
mthoilng languages and systems allow one to use 
ogrammlng languages too. When this Is the csbs, then 
being done within that language really does use a 
nguage, so the authoring Bye tern becomes lees and lees 

In the early days of Coursewrlter I looked at an 

progrnm In physics concerning Gauss's Law. However, 
the code, supposedly written In Coursewrlter, 1 
901 was In Assembly code for the IBM 1500. 
leg languages and systems are often particularly weak 
n, a critical aspect of development. Indeed, with 
ra the only possibility for revising a Bectlon of 

to redo it. The facility must be recreated on each 
, often an expensive Job, so transferability Is not 

ally a language for computer based learning must have 
m all the capabilities of a general purpose language, 
er ends up recreating a general purpose programming 
th a few additions. It makes sense, therefore, to 
e other end with a gor<v language and add a few special 
- 17 - 
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capabilities In the form of procedures. 

Many of the developers of authoring languages anJ systems 
are not highly expe lenced with learning moterlal. They often 
have a simplified view of how the computer can be used In 
lemnlm;, and the language they develop In restricted to that 
simplified view. As the view expands, under pressure of new 
materials, the language may be extended If funds are available. 
But then It becomes clumsy, as these new features were not taken 
Into account In the Initial facility. 

There Is a tradeoff, too, between how easy the language is 
to use and how restrictive It Is on what It lets the author do. 
Thus, languages that are advertised as very simple, easy to learn 
by everyone, can usually only produce the ciost trivial material. 
This will become boring even within a single long program. Given 
the future needs for slxable amounts of computer based learning 
materlalr authoring systems ere an unfortunate direction. 

I now wish to move from this discussion of the current 
situation to consider the future of computer based learning 
material. In an area where the technology Itself Is changing 
rapidly, it Is particularly Important to think ahead. 
future Hardware 

The computer, particularly the personal computer, Is 
evolving very rapidly. Computers are not only becoming cheaper, 
but their capabilities are Increasing rapidly. New systems w:ll 
not necessarily be cheaper than they are today. Vendors will 
give us so.xe balance between decreasing costs and Increasing 
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Ity. Central prorefialng chlpa are becoming more complex 
tor, as we move to larger instruction size and larger data , 

Meraory is also becoming cheaper and more compact, 
rtaln new facilities for the computer are important in 
ion with the future of education. Some are already 
le but need to be further refined or mde more available, 
are future possibilities, existing to some extent in 
h laboratories but with much Btlll to he learned. 
Ice output Is already available today but needs further 
A variety of strategies—analog storage, digital storage 
netic recreation are in u?e. Except for analog storage, 
ceg still have an artificial sound generally not desirable 
cat Ion. . Voice H important In the learning process, so 
voic 4 * output will be of nome value. Music, particularly 
Ivatlonal purposes, will be useful, 
aphlc^ is still relatively crude, although typically 
le on personal computers. We can expect much better 
s--more choices of color, resolution, shading and 
ty control and even general purpose animation as personal 
rs begin to use special graphic chips or use more powerful 
nd more memory. Text too needJ to be 'mproved beyond our 

rather crude text; choices of font and size of text 
be available to the designer of computer based learning 
1. 

tworklng, at the local level and nationally, is just 
g the stage where it 1b practlccl. For years we have been 
i about the ethernets, and they are so^ietlmef commercially 
- 19 - 
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available now! 

The combination of the optical videodisc with the computer, 
which I referred to mmy years ago as the intelligent videodisc, 
1b a curious one. Although this technology has been around for 
about a dozen years, and although many people have recognized its 
potential In education, It has been so far relatively little used 
for the development of educational material. Much of the 
material that has been developed, particularly for industrial 
use, 1b relatively crude, with very weak forms of Interaction. 
Full multimedia Is an interesting possibility, and undoubtedly 
one that will be pursued further In the future. 

Other forms of input will undoubtedly be developed. 
Painting Input Is becoming more common and undoubtedly will be 
improved. Voice Input is possible for discrete speech bub still 
is beyond our capabilities for continuous Bpeech. However, It is 
being vigorously pursued In the research laboratory. If we can 
solve the problems of voice input, we may not need it, since from 
the computer's standpoint the analysis of complex voice patterns, 
and the analysis of complex brain wave patterns are not 
necessarily that different. 
Where Hill the Curriculum ITlts Come From? 

If we are to move from our current state where the computer 
is relatively little used in learning, almost at the level of ° 
trivia, to a state where the computer in very heavily used, we 
need to ask where the new courses and curricula will come from. 
The production system outlined is intended to be adequate to 
produce sizable amountB of curriculum material of high quality. 
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publishers* and computer vendors, but the Interests of each was 
different. So such companies are often not satisfactory. 

2. The Distributors 

The Issue of who will distribute the Materials Is a 
different Issue. All the Bams "players" already mentioned are 
possible, but a glv?n group might decide to play a role either In 
production or In distribution or in some combination of the two. 

Distribution brings In questions of protection from copying, 
either through technical means or through legal means. He do not 
know If the copying of computer baaed learning material will be a 
serious problem or not, since our uperlences are no small. We 
do know, however, with other kinds of software, that a market may 
exist In spite of copying. Spreadsheet programs have sold well 
over half a million copies, although they are on floppy discs 
that could be easily copied. 

3. The Home Mark et 

In considering educational materials, It is very Important 
to take Into account the growing home market.. The home computer 
will need a variety of applications oriented programs to make It 
marketable. One of the categories In which such materials will 
be developed Is education. Products of this type will be widely 
available and will affect what happens In the schools. There may 
be products usable In both home and school. 
Educational Institutions and Organizations 

It Is impossible to Imagine a change of the magnitude 
predicted here, the computer becoalng the dominant educational 
delivery system, without anticipating massive changes In our 
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schools and universities. These changes are not likely 
r from internal pressures, as educational institutions are 
iscrvative, slow to* change. Even educational institutions 

favor of changes promote relatively trivial changes chat 
fundamentally affect the institutions. We can expect, due 
current severe problems in education, including both 
al and public opinion difficulties, that very strong 
I pressures will encourage change, particularly regarding 

of the computer. These changes will be particularly 
ated in the United States if such radical systems as the 

system come into widespread use. 

already soe-a-number of -non traditional ways of organizing 
Itles. The moat interesting arc learning at a distance 
tiona, such as The Open Onivetsity in Great Britain, 
existing institutions will be changes, particularly 

rfay the courses are run. The notion that a course should 
fixed "amount of time will probably vanish as flexible, 
jail zed computer based learning material is becomes 
Le. Computers will be more used in schools at all 

Changes in grading systems, made possible by the fact 
;h more is known about the student and the student's 
3, may well happen, shifting from norm-based grading to 
icy-based grading. 

Is quite possible that with the new learning materials 
111 be a shift in the aren a of education. That is, less 
>n will take place within formal schools, and more will 
ice either in public environments, such as public 
- 23 - 




libraries or the home. The learning at a distance institutions 
already represent a move- In this direction. Not all education 
will move out of the school, but certain components are <)oo<H 
candidates for ouch a move. 
T he Future—Good or B ad? 

Although it seems clear, for reasons not fully discussed, 
that the computer will become more important in education, it is 
not yet clear whether the computer will improve education, 
helping to overcome some of our current problems, or whether it 
will lead to further deterioration in the educational systems of 
many countries. The effectiveness of the use of the computer in 
education may be an important factor in determining which 
countries will succeed in the future. 

There is nothing magical about computers. Like all 
technological developments, the computer itself Is not good or 
evil. R&f- ^r It Is the way it is used by humans which is the 
critical factor, in education or elsewhere. 

The poor quality of mu*ch existing computer based learning 
material and the widespread acceptance of this material by 
teachers and administrators gives us cause to worry about the 
future. If publishers distribute third-rate material and this 
material is eagerly gobbled up, publishers have no incentive to 
develop better material. It is critical to educate teachers to 
understand the qualify of the materials, to look at computer 
materials critically, just as they might look at other kinds of 
educational material. 

This material is "reviewed" to some extent today. Most of 
- 24 - 
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these reviews are In magazines which are heavily dependant on the 
vendorr for advertising. So it is not surprising that most of 
these reviews are positive. I remember seeing a review, for 
example, recently about a new Pascal. The person had Just 
studied Pascal and had taught BASIC for years, hardly the person 
one would want to review Pascal. 

If ve are to move toward a better future in education, we 
■ust look at new ways of organising our entire educational 
ayntam. Yet relatively few such vlsione of the future are 
available, and the vast majority of teachers with computers have 
no view of where they would like education to move. 

One view that I recommend to readers is in George Leonard 1 s 
book ' Education and Ecstacy. Two chapters portray a school at 
the beginning of the next century. While the technology that 
Leonard portrayed has been to some extent surpassed, because of 
the quick rise of the personal computer, the educational ideas 
are worth looking at. 

This presents only one view of the future. It is hoped that 
readers will develop their own visions and that they will take an 
active role in moving us toward a b«tter educational system. 
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The Educational Technology Center Is internationally known for 
its development of highly Interactive graphic computer based 
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from Digital Press (Digital Equipment Corporation, 1981) and of 
forthcoming Personal Computers in Education , to be published by 
Harper and Row, in addition to many articles and other books, " 

This paper concentrates on two examples of computer based 
learning, both developed at the Educational Technology Center at 
the University of California, Irvine. Both are intended for a 
very wiue audience, but have been primarily tested with junior 
high, high school, and university students* Both sets of nodules 



cnnc^n important educational problems, problems which have 
proved difficult to solve In any other way in our educational 
system. The products are essentially complete) all the material 
can be inspected on developmental systems. We are currently 
moving these units to delivery systems, 

The two projects addresB scientific llterAcy and with aiding 
students to become formal reasoners . The first is supported by 
the Fund for the Improvement of Post-Secondary Education and the 
second by the National Science Foundation. 

Both projects, in addition to extensive teeing in schools, 
have also been tested of public snvironments, principally in " 
public libraries and science museums. The modules are designed 
to run entirely self-rtahding, with no teachers, print material, 
or other aids- available. In the libraries, the computers are 
simply, sitting there and library visitors come up and begin to 
interacv with the computer. 

Thus the materials must maintain the Interest of the 
viewers! they must be be strong motivattonally. They do not 
requi're^ijrBpecta-l- knowledge about computers, as anyone coming 
into the library Is a potential user. "Furthermore, they must not 
h«e any hidden assumptions about what is necessary to understand 
the subject utter areas but must accept all students where they 
are. 

. A typical module lasts, with th» average student, about 90 
minutes to two hours. Modules are divided Into a series of 
sub-modules, usually 8 modules of about 10 to 15 minutes eac*\ 
The units are coded In udo Pascal. The programs are highly 
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interactive. The average time between interaction In our 
programs is approximately 15 to 20 seconds. Thus the student is 
always being asked to do something, therefore the situation is 
extremely different than that of a book, or a lecture. He are 
also concerned about the quality of Interaction in oui programs, 
nus we nevir ue multiple choice, as. we consider multiple choice 
to be a very low quality interaction, even in testing. 

Considerable consideration is given to the visual appearance 
of both the text and graphics on the screen to aid the learning 
process. Thus, large amounts of blank space are used, lines are 
short, lines end with natural phrases, and no hyphenation is 
used. 

The production system that led to these materials follows 
that which has been developed for many years at the Educational 
Technology Center. Further descriptions of the production 
strategies are available on request, and it is described in 1 
considerable detail in the two books by the author mentioned in 
the bibliography (editor: are these bibiligraphies really to be 
printed with the paper or should we repeat the references here?]) 
SCIENTIFIC LITERACY 

Our society is heavily dependent on modern science and 
technology. Yet our educational system does, at the .present 
time, an Inadequate job of helping the vast majority of people In 
our society to understand what science is all about. It is much 
easier, both In the schools and the universities, to deal with 
the (actual areas of science, to expound the laws, and to teach 
people to solve problems, than It 1b to give any overall feeling 



as to the nature of the scientific enterprise. Only a few 
courses have attempted to do this, such as the high school 
Project Physics course developed at Harvard University. Host 
courses in the sciences give only lip service to getting students 
to understand the nature of science. 

He view this to be an important national problem, one that 
is not being addressed effectively In our current educational 
system. Some of the earlier atteir 'ts to address the problem of 
scientific literacy In the general population, such as seen in 
the Elementary Science Study Program, Science Curriculum 
Improvement Study, and In Science, A Process Approach, have now 
little use in American schools. 
The Educational Technology Center realizes the importance of 
this problem and for a number of years sought funding to develop 
computer based learning materials to deal with this important and 
difficult isBue. Finally we were successful In persuading the 
Fund for the Improvement of PoBt-Secondary Education to support 
the project. They were interested partially as an adult 
education project, because of the use of the materials In the 
public libraries. But I believe they were also attracted by the 
potential of the same material serving a very wide audience from 
young children through nature adults. 

Although the various units work in a variety of subject 
matter areas, the general tactic in most of the units is the 
flame. He attempt to place students In an environment in which 
they must behave like scientists in their approach to problems. 
llBually one or more of the early nodules starts with the 



gathering of empirical Information through built-in facilities in 
the program. These experiential controllable worlds (similar 
facilities are called mlcroworlds by some authors) allow learnerB 
to play with the -^..obom,.) Involved and develop some ev 
Then, based 0,1 this, we encourage the development of hypo; a 
on understanding the evidence, 

It is critical that the scientist not only observe but also 
develop models and theories which account for the observations, 
In this model-building activity within the dialogs, the Btudenti 
receive considerable help, He want every student to succeed, 
provided that an honest effort is made. As hypothesis are 
developed, tentative models, a student can then test these 
against the empirical situation. It nay be that the hypothesis 
will need to be modified, 

In some cases, we •mislead" students so t'lat they will 
Initially develop "false" hypothesis which account for the 
evidence already gathered, but will not account for future 
evidence. Thus we try to give some of the flavor of failures of 
science, as w* 11 as the successes. 

As the theory or model is developed to account for the 
evidence, the emphasis shifts to the predictive power of 
scientific theories. The students must make predictions using 
the theories that have been developed and these predictions are 
compared with the experimental situation. 

In most cases the apparatus is simulated on the computer. 
He hope later to do some detailed studies of the effect of actual 
apparatus versus simulated apparatus. He considered this 



question carefully in the early stages of development. In 
viewing the earlier projects which had not succeeded, one of the 
problems was that, regardless of the desirability of 
undesirabU, .pment, many teachers did not want to use 

the equipment, a,, indeed, refused to make use of the kits of 
apparatus. 

The presence of the jquipment would present logistic 
problems in libraries, or in what we regard as perhaps the most 
important eventual use of these materials, in home use on 
personal computers. Some decided, in most cases, to simulate 
the equipment. 

Some of the ideas involved in the earlier curriculum 
projects were very useful to us. Thus, we developed a Batteries 
and BLilbs module following quite sirllar strategies to that 
developed in other discovery oriented projects in the '60s, -He 
were able to profit by bringing in teachers and developers who 
had extensive experience with the earlier curriculum materials. 
Some of the modules concerned actual scientific situations, but 
in other cases we deal with Imaginary situations, theories which 
do not exist in nature, or theories which exist in nature but 
that the "evidence" we esent is not actual evidence but 
fictional evidence. 

*s m example of this last type, I will describe briefly one 
of the modules, Tribble Families. This module starts with 
fantasy. The student is the chief scientist of a spacecraft, 
which descends on a strange planet and finds a number of peculiar 
looking animals called Tribbles, in the first unit of the 



program, the task is to classify Tribbles, noting differences 
between Tribbles. The Tribbles, unlike any animals seen before, 
differ in only three characteristics: eye color, size, and horn 
shape, In the second module, the task become* that of finding 
out which Tribbles, when put together, have children and which 
don't. As with everything else in these modules, this topic must 
be approached purely empirically. The students are never "told" 
things but rather roust gather data by putting Tribbles together 
and seeing what happens, then they try to generalize on the 
results that they see. 

Eventually, Trlbble f ami 2 lea' proceeds to develop a Mendelian 
genetics theory. The approach is /ery different than that in a 
typical biology class. The students are not told the underlying 
theory but rather it comes out of study, somewhat the way a 
scientist might do it. We cannot, of course, be fully realistic, 
but we can, and we believe we do, give the flavor of wlut It is 
like to work in science. 

The project manager for this project was Barry Kurtz, 
FORMAL REASONING 

The second project is concerned with the recent finding that 
many students in the United States do not become formal 
operational In the sense of Jean Piaget, at the age, 
approximately 12 to 14, where Piaget says it should be happening. 
Particularly recent studies in different parts of the country of 
college students, using the Piaget formal operational tasks, show 
that as many as half of the students in universities, either at 
the freshman or sefiior level, are not formal operational, yet if 
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one examines the content of many freshman courses, particularly 
In the sciences, we see that the formal operational traits, such 
as ratio reasoning, are heavily used in these courses. 

As in the previous project, my colleagues and I at the 
Educational Technology Center saw this as a serious national 
problem and one that was not likely to be solved by the 
conventional classroom methods. It is our suspicion, although I 
do not know of very much in the way of formal evidence, that many 
teachers are themselves not formal operational. So it is 
unlikely that these teachers will be likely to help students 
become formal operational. He saw some possibility that the 
computer could help with an important problem that was difficult 
to solve with traditional learning methods. 

Each of the modules is based on one of the Piaget formal 
reasoning tasks. Each involves some subject matter area in the 
sciences or mathematics, so the materials could be used within 
science or mathematics classes. Experience before computer 
material has indicated that a single subject area is not 
sufficient for the student to develop a concept. That is, if 
they learn, for example, about ratio reasoning in one context, it 
does not necessarily carry over into another context Involving 
ratio reasoning, 

Hence, in some cases, we have been able to develop several 
nodules in different areas. Thus, ratio reasoning in represented 
in this project with two modules, one based on the concept of 
speed and one based on the concept of density. Two will will not 
be enough for our purposes, but as with all projects, we had a 
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finite amount of resources to work withl 

As an example from this project, I consider the speed module 
just mentioned. This module was based on earlier work of David 
Trowbridge for his doctoral dissertation at the University of 
Washington. The first sub-module is again experential, with the 
student allowed to fire balls in different ways with pinball-like 
mechanisms with particular tasks assigned in the process. The ' 
notion is to build up some intuitive conception of speed, 1a the 
next module, we consider situations in which the time is held 
constant but the distance can vary. Then we consider the 
opposite case, situations in which the distance is held fixed 
because the balls travel in tunnels of fixed size but the time is 
variable. In each case, students are presented with large 
numbers of examples and must decide, given pairs of motions, in 
which cases the balls move faster or slower. 

In later sections of the dialog, both space and time are 
allowed to vary so the true ratio idea enters. Facilities are 
provided tor both space and time in some of these modulen. 
Project Manager for this project was David Trowbridge. 

CONCLUDING REMARKS 

These two projects illustrate some of the developmental 
activities of the Educational Technology Center. We have, at any 
given time, a range of research and developmental projects active 
at the Center. Those interested in our current projects are 
invited to w * for further information. 
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1, The real problems with computers In schools are toe of 
getting enough hardware , for the schools-enough computers for 
each school . t 

The number of computers in schools has been doubling or 
tripling each year for the past several years, In spite of the 
stringent financial conditions of schools. He now have 
approximately 300,000 computers in schools, averaging about three 
per school. Although numbers are increasing rapidly, the 
sophistication of usage within schools Is often primitive. 
Teachers are often 111 prepared to use the computer. Very little 
good Instructional itcUrlal is available. Teachers have low 
standards for selection of instructional material. Inadequate 
programming techniques are often being taught, crippling students 
for later programming activities. 

This is not to say that some good uses do not occur in 
schools. But on the balance it may turn out that at the present 
time the students are being hurt more than they are being helped 
by the computers in schools. More machines will only compound 
that problem. 

2. / the computer Is used In classes, or other learning 
nvlronrpgiK, is good. 



The computer, like any other learning aid, can be used in 
ways which are helpful to student learning or ways which are 
harmful to student learning. There is nothing magical about 
having a computer. 

A computer with poor curriculum materials, or teaching the 
material in Inadequate ways, can be devastating to students. 
Likewise, teaching inadequate programming techniques, no longer 
used in practice, can hurt student's progress rather than aid 
students. The use of any educational device in the classroom 
must be evaluated on Its own merits, regardless of what kind of 
learning technology is used. 

3. The computer can be used In only one reasonable way in 
classroom environments . 

Adherents of various points of view often have this belief. 
That is, they believe that other ways of using the computer 
besides the one that "they" favor, are intrinsically wrong. 
Decisions about how to use computers for learning should be made 
entirely on pedagogical grounds, not on predetermined 
philosophical positions. 
' <. All students should learn to program . 

As computers become more and we wi^spread in our society, 
the vast buV o be i, use cf very powerful 

■Mite? dlectual tools. Our current tools, such as 

word processing, spreadsheets and personal data bases, are only 
beginning to show us the real power of such tools. Because of 
these tools, most people vill not be involved in programming 
itself. 



Progcamniing is often justified on the assumption that it 
helps to give students some idea of what computers are all about! 
But this is not true, Host introductory programming courses, 
even extensive courses, do not give a full picture of what is 
possible with the computer, Often the applications tools, just 
suggested, give t much better idea of the real power of the 
computer. 

Because Ujs easily available on small compters, we should 
teach students BASIC . 

One of the major disasters In current schools Is the 
teaching of BASIC. Almost all BASIC teaching in schools is 
developing a sjeries of very fundamental bad programming habits 
that are difficult for students to overcome later. Host students 
who study BAS'C in high school have difficulty with computer 
science in the university, c* In any other modern programming 
applications, 

6. We know all about how' to use computers In education . 

Our knowledge of the learning process is weak in many 
respects* Conflicting theories of learning still abound. 
Furthermore, much U still to be learned about the computer 
itself as a learning device. Additional research is needed. 
This is not to cay that we should, stop doing any development 
until the research is complete. In fact do not believe that 
we can afford to do that at the present wi-;, perhaps because the 
research may not be complete for hundreds of years. 

Teachers cin develop extensive curriculum material in thelf 
sr are time . 
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A legend has sprung up buggests that teachers will be able 
to develop large amounts of computer based learning material. 
Indeed, the programs in many schools are predicated on this ' 
unlikely assumption. The development of any type of good 
learning module, involving the computer or any other learning 
media, U a process that requires much effort and care. Hence, 
very few people, including teachers, are likely to be able to 
develop extensive computer based learning materia; in their spare 
time. 

8, If a program draws lots of pretty pictures, it must be a 
good program . 

Although I have long argued for the importance of graphics 
In education, some recent learning programs are almost nothing 
but a set of pictures, unrelated to the learning material. While 
visual information is important in learning, the developer must 
work hard to make certain that the visual information is an aid 
in learning and is not just a gimmick* 

9, Authoring languages and authoring systems are useful in 
producing computer based learning material . 

Host authoring systems and authoring languages intended for 
pedagogical design are toys, allowing the production of only 
restricted types of computer based learning material. 
Furthermore, they tend to confuse the issues of pedagogical 
design, with the technical details of programming the material. 
More adequate authoring systems exist. Software to aid the 
coder, on the other hand, can be useful. 
10s He cannot write useful computer based learning material 
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until a rtificial Intelligence techniques are practical for 
general classroom use . 

Although some of the techniques of artificial intelligence 
will probably eventually demonstrate their usefulness in computer 
based learning material, we can filready develop, without such 
technique*, highly effective learning material dealing with very 
difficult learning problems. The computer is at least as 
intelligent as any book, even in its least intelligent form, and 
books have long been shown to be very effective in tho learning 
process. Programs can reflect the intelligence of very good and 
experienced teachers, without requiring built-in intelligence. 
1 1 • Since computer "Alpha" is common in schools, all curricu lum 
materials should be developed for this computer . 

The number of computers in schools tends to double or triple 
each year. Furthermore, new models are appearing constantly, go 
any curriculum effort which bases its thrust on computers 
currently in schools is soon likely to be in difficulty. It is 
essential to look ahead. 

12. Computers will certainly improve education . 

As already indicated, the computer is not necessarily a 
desirable force in education. Computers can lead to a better 
educational system, or to a worse educational system than we have 
now. It is only that if we insist on the highest quality, both 
commercial and non-commercial products, that we are likely to 
improve education with the computer. The issue at the present 
time is very much in the balance. - 
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ABSTRACT 

This paper concerns the role of graphics. in computer based learning material. We 
consider both textual^d graphic components of screen design and both spatial and 
temporal aspects. Informal design standards are developed. Several software approaches 
are discussed. 

COMPUTERS AND OLDER MEDIA 

The computer's use as a learning media is just beginning. It promises major changes 
in our educational system. Every new learning mode, suph as the \computer , has unique 
properties. This paper addresses those aspects of learning via computers that are 
particularly concerned with spatial and temporal aspects of displajkon the comouter 
screen. , \ 

The computer screen is the arena for spatial and temporal display of information in 
computer based learning for both the computer and for the user. (We maV also have sound 
information.) On the display we may find both iconic (pictorial) components and textual 
components. The way information is placed on the display can have a great effect on 
learning. 

The computer as a learning medium is very different from older media such as books 
and lectures. Thus, the computer screen is very different from the page of a book. 
Successful curriculum develooers in older media treat the new medium just like the older 
one, carrying over successful tactics from previous use: the screen is often formatted 
like the page of a book. \ It is only by* conscious effort that we begin to understand the 
great differences betweenVthe computer display and these older media. Some differences 
reflect different capabilities. For example, as soon as^ reader turns the page of the 
book, the entire information on the new page appears at once. The computer display is 
extremely different? information can develop in time , under control of the computer and 
the user. * 

*v A second difference concerns cost. Things which are possible in new and older media 

may be practical in one and impractical in another. Thus, while sophisticated designers 
understand that blank space in books aids learning, the amount of blank space in books is 
rigorously controlled for financial reasons. One cannot afford to put only a few words 
on each page, even if that were tfee most effective way to promote learning, because the 
cost of a book increases as the amount of blank space increases. However, with the 
computer screen no such relationship exists between the amount of blank space and the 
cost. Hence, blank space, while possible in both, can be used much more freely with the 
computer . 

Even a cursory review of much current computer based learning material shows that 
the developers often are blissfully unaware of thr differences between bodies arid computer 

^displays. The usual procedure is to start at the top left corner and to put the . 

v information densely on the screen at the maximum possible speed. The computer "pages" 
thus resemble the pages of a book. Few authoring systems give developers any capability 
for better control of this medium. 

Often decisions are made purely on a techological basis. Thus, many developers of 
computer based learning material appear to believe that the faster the information 
appears on the screen, the better. This does not match our experiences at the 
Educational Technology Center with young novice computer users. We suspect that fast 
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output reflects a desire to use the technology to make the computer presentation look 
more like a book presentation. 



CHANGING CAPABILITIES 

In dealing with the computer as a learning medium, "e must recognize that its 
capabilities are improving rapidly with time. Some of this improvement, such as faster • 
and cheaper processors and more memory, simply refers to general computer capabilities. 
These factors will influence the effectiveness of the computer as a learning medium. 

It is a fundamental mistake to consider today's small personal computers — the PET, 
the Radio Shack, or tKe Apple — as typical of personal computers in the very near future. 
Any realistic consideration of screen characteristics must look into the immediate future 
and must make some reasonable assessments as to what will soon be available. 

Developmental strategies must take into account the changing nature of the medium. 
Thus, the development should be done in such a way that it is reasonably easy to 
incorporate into new versions of the learning units new capabilities not available at the 
time of development. The programming must ensure that it is relatively easy to modify 
the code? the question of hov; to program in a dynamic environment where changes are 
expected is the major issue. Indeed, in most major applications areas the cost of 
maintaining and revising programs over their life cycle is likely to be greater than the 
cost of initial development. The criteria we need to consider are the same criteria used 
in large-scale complex programming. 

We can delineate areas where we can expect screen capabilities to be improving in 
the near future. Consider first characer istics of screen graphics. Current graphics on 
inexpensive personal computers are limited by poor resolution, low speed, inadequate 
numbers and control of color, the lack of shading and intensity control, and the severe 
limitations on animation. We can expect all these to improve. Several technological 
factors are important in this improvement. First is the separate processing unit to 
handle the display. These processing units may be designed for graphic capabilities, 
such as the recently announced NEC chip. It is reasonable to expect competitors. 

Another factor in the rapid improvement of visual quality is the decreasing cost of 
memory. Screen resolution in a raster system, storing bit maps, of the screen, can 
improve as the cost of memory comes down until the limits of current video tube 
technology are fully realized. Beyond this point we need inexpensive, higher resolution 
monitors. This may be a slower process, because we cannot piggyback on the large TV 
market. 

As increasingly powerful processors stop using low quality monitors to control the 
display,, the importance of languages which allow multitasking will increase. Many 
graphic activities could be improved by using separate graphic processors. While some 
aspects of separate use can be handled without tasking capabilities within the 
developmental language, to make full use of separate processors it is necessary to start 
separate processes and have them proceed independently. 

Character display capabilities will also improve, although not as quickly as 
graphics. Currently computer users have mostly an abominable set of characters on the 
screen, characters which would be considered completely unacceptable in the print medium. 
The main concern of those designing character sets seems to have been to make certain 
that an "A" is distinguishable from a " B » Thus, hundreds of years of print character 
technology has been lost in most computers. 

This situation is changing. We now have experimental systems capable of different 
character fonts, and we also have systems which allow user choice of characters. We are 
also beginning to see systems that allow choice of size and'color of characters and other 
similar features. The character problem is connected with resolution, already raised 
with respect to graphics. We may expect major improvements in quality of characters to 
come from Japan, since Japanese characters require better resolution than those in 
western alphabets. 

A third area for improvement is the development of new and better forms of input and 
output. Thus, we are very much looking forward to greater availability of voice output. 
Voice output, t such as that available from Texas Instruments, is rare on personal 
computers. High quality music is available but also rare. Several current projects are 
considering voice input, but none can be expected to reach full fruition for personal 
computer use for many years. 

Personal computers are often weak in pointing devices, and additional work and study 
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p in this arec is needed. Some widely touted pointing devices, such as the finger, have 
relatively poor resolution. Others are difficult for the novice to use. 

Based on these future considerations, we can ask about reasonable design standards 
for computer based learning. Our design standards will .include both the spatial and the 
temporal aspects, since both space and time are controlled by the designer and (possibly) 
the user. 

REASONABLE DESIGN STANDARDS 

The design standards specified in this paper are based on extensive experience at 
the Educational Technology Center in developing a large amount of computer based learning 
material, both initially in the timesharing mode and more recently with personal 
computers. During the fourteen years in which our group has been active, we have 
considered design issues very seriously. Recently we have been fortunate to have funds 
for experiment*] studies from the Research in Science Education program in the National 
Science Foundation. 

The first thing to be said is that design standards for computer based learning 
material should be the standards of good graphic design in any application. 3y good 
graphic design we mean the standards that would be followed, for example, by an excellent 
designer in producing an ad for a high quality magazine, but 3S appropriate to the new 
medium. The necessity for such standards in computer based learning is not generally 
recognized. Nevertheless it is important in effective use. A corollary to this comment 
is that a professional designer should be employed in the development of the highest 
quality computer based learning material. We consider later the environment in which the 
designer should work. 

We can formulate a basic general design principle, applicable to all the material 
that follows. Since we are concerned with learners, we want the learners to focus on 
what is important, whether th is . information be visual or textual. The implication is 
that we should remove unnecessary information from the screen / and only show at a given 
time material important for the learner at that time. We do not need to follow this rule 
slavishly, but it furnishes a good guiding principle for the developers and for the 
graphic designers. 

ICONIC INFORMATION 

One aspect of what appears on the screen are the images, the pictures to aid 
learning. Here too we can state a general principle. We want to use as much 
educationally relevant visual information as possible . Much of the learning experience 
should be carried by visual information. This message is one that the pedagogical 
designers as well as the graphic designers must keep in mind. H< 'undancy — information in 
both forms — is desirable. 

The tendency unfortunately is in the other direction. Since many curriculum 
developers are the successes of our current highly verbai educational system, they tend 
to be verbally skilled, more skilled than many students. Therefore, they often do not 
realize the necessity of good pictorial information for the average learner. Many people 
who are "failures" in our current educational system could be successes in a system which 
paid more attention to the uses of visual information to aid learning. 

It is still not easy to satisfy this need for inconic information. We must keep 
focusing the designers' attentions on pictorial design. Material that is heavily verbal 
can be sent back to the pedagogical designers, prodding them to give more consideration 
to visual components to aid learning. On the other hand, pictures should be relevant to 
the learning process , hot just pretty pictures. ~~ 

Several features of the use of pict ( \ information are unique to the computer, as 
compared with books. Some are devices w. 3 re well-known on the blackboard or on TV. 

For example, it is not necesary for pictures to appear instantaneously. Mater ial 
can develop in time . Often the text and pictures can be interwoven with parts of 
pictures coming on in response to specific references in the text and with suitable 
delays in between. Timing can emphasize critical features of the pictures by delaying 
before and after those critical visual features. We can also highlight these critical 
features or make them stand out in other ways such as blinking. 

Another factor derived from studies which preceded computer use is that a picture 
can have too nanv details . That is, details which distract the student from critical 
features can cau.se learner problems. In any picture we must identify the critical 
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features and ask how these can best be brouuht out by the way a oicture is Drr>o * 
ask if a given detail is really essential, or whether it can be omitted, e -cnteil. Wr 

TEXTUAL INFORMATION 

The pictures and text cannot be considered separately. They both contributP hn 
learning, working together. ° 

Perhaps the major consideration with regard to texual information is that we shoun 
use large amounts of blank space . A given screen should be relatively sparse, both 
textually and pictorialiy, at any one time. Much of the area on the screen should 
typically be blank. As indicated above, one way to achieve this is to eliminate th* 
information from the di splay no longer needed by the user . Often we see screens which 
contain a large amount of "past history," not relevant to the current situation. This is 
neither necessary nor desirable. 

A second consideration is that text should be placed on the display with the same 
care that would normally be used in placing a picture on the screen. Text placement 
shouid not be a system accident. Software for computer* based learning is poor if the 
system decides where text is to go and how it is to be displayed rather than the. 
pedagogical designer, the graphic designer, or the learner" making this decision. 
Conscious placem ent of text , allocation of display information, should be considered at 
all stages in development. 

Strategies to increase comprehension with print material are known. Although little 
definite information about these strategies refers specifically to the display, it seems 
Leasonable that previous print readability research will be applicable to screens, Thus, 
short lines, requiring less eye movement in reading a line, are generally desirable. 
Avoiding hyphenation is highly desirable—hyphenation is a barrier to comprehens ibili ty . 
Keeping natural phrases together on a line is also another important consideration; the 
phrases can be defined syntactically or semantically where the experierced reader will 
pause. Justification at both ends of the line is also undesirable. 

A variety of methods, again both spatial and temporal, are available for stressing 
material on the screen. Thus, there can be a pause before and after a critical word or 
P hraSe or the word or phrase can blink or it can appear in bold or in some other way, 
such as an alternate font or color, that sets it off. 

An important consideration, unique to the computer, is the quantitv and degree of 
viewer control allowed in the display text. In all of our current programs we allow the % 
user to choose at any time the rate of text output. The decision followed by young 
students is often different from that provided by most developers of material — the 
typical person not acquainted with computers prefers a much slower rate of text output 
than is usually seen. ~" 

Students can control not >nly the overall rate of text output, but also the timing 
between words and other aspects of timing. Furthermore, many of the features stressed in 
the last paragraphs also can be turned over to the user. The question of which of these 
is most important for increasing comprehension and motivation needs additional 
investigation. A current research project at the Educational Technology Center is 
considering learner control. 

To maintain interest we can vary the stvle of display with size of the tvoe, fonts, 
and varying alignment or justification procedures So the learner doesn't see" the same 
old thing repeatedly. Varying style has strong motivational components. Our software 
for text output at the Educational Technology Center supports three justification modes: 
Left justification (without right justification), right* justification (without left 
justification), and centered text, the text centered on the line within a particular area 
of the display. We use specified a.eas, ports or windows, with a strategy similar to 
that used for display of graphic information. 

For student text input, our group borrows from good word processing practice. Text 
input is always in a "port," an area on the screen as just described in connection with 
output. The port for input may or may not be the output port. The software handles 
wraparound like word processors? if students type a word too long to fit within the port, 
they see it on the current line until the edge of the port is encountered. At that 
point, without any fuss, the partial word is moved to the next line and students can 
continue typing with ro need for any special action. 

Timing is also important on input. We want a user to be reminded of what to do in 
many situations ("Don't forget to press return!") or given other specialised time- 



?!n?nn • p * the other hand, the software should not cut off someone currently 

EHura?!nna! t • * sizable delay preceded this typing. Again, these factors are in the 
compute? Tecn,1ol °^ c * nc * r software and so arc in all of our programs on the personal 
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Several individuals can be involved in design. The visual design components come 
from at least two groups in the design process. In. the pedagogical design component 
excellent teachers, curriculum designers, and others work closely tooether to develop the 
necessary pedagogical specifications. They must be concerned with any factors which they 
believe will influence learning. Among these factors are the factors of design in both 
space and time. * 

We encourage pedagogical development teams to specify any design details they regard 
as critical for use. Of particular importance, often determinable only by someone with 
time 13 expert ^ se ' 1S the <3 u estion of what needs to be on the screen at a particular 

The pedagogical design team specifies design details in two ways. First they may 
draw sketches of the screen, showing approximately where oictorial information and 

H!ct^nfrc yPe ?i° f . t6Xt u al f*f°™iation «e to appear. In some cases the pedagogical 
designers will give this information in considerable detail. In addition visual 
in enf^ 10n ' scripts produced, can also contain extensive verbal instructions for the 

visuax designer or for the coders, timing considerations or information about where. and 
how things are to appear on the screen. / 

We run a half-day, informal workshop for our authoring groups, before the full 
pedagogical design process. One of the components stresses the display design details 
that form the oasis of this paper. 

The amount of screen design input which ccmes from the oedagogical designers will be 
var able. Some individuals are very conscious of these issues and will give them- very 
careful specification, other groups will almost entirely ignore these issues. So the 
quality of visual and temporal design at this stage will vary. 

The next person who may be involved is the graphic designer, an individual with a 
good design background in other media. This individual also needs- to understand the 
educational implications of design, implications which often are not part of the standard 
design curriculum. So it may be necessary to train this cerson. Unfortunately we are ' 
not often in a position to afford a graphic designer. Much* of our materials have had to 
rely only on general design principles as stated rather than on the insight of a graphic 
designer. As we move toward more large-scale development of computer based learning 
material, graphic design will be a critical feature of the full-blown production system, 

. DESIGN SOFTWARE . 

If graphic designers are to function effectively, they should be working on-line, 
creating in an easy-to-use system the actual displays that the viewers will be seeing 
insotar as these displays can be determined before user input is known. 

If the designer is to work interactively at the screen, soecial software must be 
provided. We have developed a variety of such software at Irvine, although we are still 
not satisfied with it. Several systems developed at the Educational Technology Center 
will be described briefly. 

The first graphic design language was developed for timesharing, initially as a 
picture editor by student programmers working with a program where manv pictures were 
necessary. It evolved into a general display editor, used extensively' for several years. 

This facility, a series of APL functions, allows users to create, modify, store, and 
delete, objects, either text or pictures. All objects can be referred to by names. 

The user interactively constructs objects needed for a display, placing them in the 
desired locations. They can be easily moved. When the design is complete, it is "oasted 
down', just as a graphic artist might do in developing an ad for a magazine. While'most 
objects came through functions, tablet input is available cor drawing objects. 

The program is a code generating program, writing commands in the macro capability 
that we used in our production of computer based learning units. It stocod the full 
graphic data, since we were working on a timesharing system with adequate file space. It 



wrote either in-line code or three types of procedure calls, depending on whether the 
object's position was known at the time that the designer was working or whether the 
position depended on choices known only when the final dialog v;as actually run. 

We also developed learning sequences to aid people to become familiar with the 
facilities. The most successful was in print form. 

When we shifted from timesharing systems \;o personal computers, we created tools to 
support efforts on these smaller machines. in this area, we initially developed three 
tools: a text display package, a graphics display package, and a graphic design system. 
The last was intended for the designer. 

The text display and graphic display package are similar. Both support the goals of 
effective screen design, transportability, software maintainability, and simplicity of 
use. The vocabulary and programmer interfaces for the two are consistent and new 
programmers are able to use them effectively after a single introductory course in 
programming. These packages provide a uniform and "friendly" user interface where the 
display of information and handling of input is controlled in a manner dictated by sound 
instructional and graphic design. The graphics procedures were motivated by the ACM 
SIGGRAPH core graphics standard. 

The graphic design system allows definitions of the position of ports and objects to 
be displayed. The design of the system is based on three concepts: the importance of 
screen layout, the hierarchical design of displayed objects, and a hierarchical display 
interface. 

Most graphical objects have naturally hierarchical composition with "natural' 1 sub- 
objects. Eventually one arrives at primitive constituents, such as points and lines. 
This hierarchical structure facilitates creating and editing of graphical objects. 
Pictures can include other objects, which may be selected from libraries of previously 
defined objects , called "sketch books." \ 

Users are presented with choices from a tree of menus, graphics related functions 
together. The cost is that one must know something about the overall organization to 
know where a particular function is located. To alleviate this problem and that of 
possibly needing the same function at different places, commonly used functions were made 
available from multiple places. 

To protect from mistakes, designers can "back out" or undo most actions. For 
example, when an object is edited/ only a copy of the object is changed until, after 
viewing the fully edited object, the user requests that the changed version be "saved." 
The ability to "back out" is implemented at every level. Development is still 
proceeding. 



Design is an important component of good computer learning secuences. In this paper 
we have suggested some design standards. 
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is an exciting era Tor those of us who have been 
r many years with computer based learning.. We are 
to see commercially available products for aiding 
d learn via computers. Many of these products are 
Unfortunately most (perhaps 90%) of these early 
2 very poorly contrived. 

is early stage of the market, when almost any product 
35, quality has not always been a determining factor in 

people to market a product. Indeed, some marketers 
f beginning to distinguish between their first 

materials and their better scjond generation computer 
ling materials. 

, however, often misunderstand this distinction and 
tinguish the better materials. If we are to demand 
:s market higher quality material, users must develop, a 
idards for examining material. This paper Is directed 
development of such standards for all users. He must 
jnderstand that any old learning material on the 
i not necessarily "good." I hope that my comments on 
:erial will be uneful for developers of computer based 
iterlals? I discuss development In more detail 
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elsewhere. 1 

The current situation, the poor quality of material 
commercial available, has great inherent danger. The problem is 
that the implicit inadequate standards promulgated by these poor 
materials may become Uie standard for computer based learning 
modules. If this hpapens, we will lose, perhaps for a long time, 
the real benefits of the computer in education. ^ 

My tactic is to enumerate practices in computer based 
learning which are common but undesirable, often these are 
practices which the novice does not note, but which are 
nevertheless worth discerning. \ 

I will avoid pointing to specific examples from existing 
material, Dut readers may find it Interesting and profitable to 
examine. such examples by applying the ideas developed in this 
paper. 
Content 

Undoubtedly the most important consideration in judging any 
learning material is content . We are interested in material that 
aids learning, so that should be our primary focus. Is it 
substantial material, Intended to accomplish some important 
learning function? Does it accomplish this function? A 
surprising amount of material, both comptuer and othei , is very 
vague on issues of content; it is often unclear how learning 
material fits into existing or proposed new sequences of 
learning. 

The iiiont sorioun weaknoanea in the contniU are what 1 shall 
cal1 trivial pieces of material, learning material which is 
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Inconsequential. Limited segments of learning material, computer 
or otherwise, need not be inferelor, but often is. A small 
amount of material may surest a very exciting new approach to a 
particular area or to learning more generally, But the small 
computer programs currently avallble for learning seldom do that, 
Indeed, they seem to have been developed primarily because the 
materials' was "doable/ easy and cheap to program, not because it 
hafl any serious luarnlng purpos*) time, technology rather chan 
education dominates, 

One typo of of fuller U die mifltw "game/ At least two 
problems exist, Hm i thU Umilnluyy U applied to Almost 
anything, The ward 'game' 1 is used to apply to activities that # 
as far as I can see, have no game-like aspects, assuming the 
everyday meaning of the word "game/ 

Another problem is that games seldom tie in well with 
specific learning situations. What is to be learned in playing 
an educational game is often vague to both student and designer, 
If they are true games, they often are strongly motivational. 
Although motivation is certainly extremely important in learning * 
It is not the whole story. 

One could achieve similar motivation in some cases by 
bringing a plnball machine^ into class, One could then argue 
profoundly about how the students are Intuitively learning 
information about the laws of motion in mechanics, particularly 
with regard to colliding objects? university faculty are skillful • 
in making such arguments. I think these arguments would be weak, 
for sustained use of pinball machines, Educators need a clear 
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view about what is motivational with little educational value .arid 
what is motivational with strong educational value, Materials 
should be related to learning. 

Games, I hasten to emphasize, can be extremely important in 
the, learning process, more important than they are today. We 
must consider motivational issues. But we must develop games 
very skillfully If they are to incorporate learning issues. We 
"cannot assume that anything called a "game" is automatically 
valuable, 

Another area in much existing material where content in the 
learning sense is particularly weak is the type of help offered 
tu students in difficulty. This is" perhaps the most critical 
capability of good computer based learning material. Good 
students typically do not need such material. They can learn 
with almost any media, The real advantage of computers entails 
being able to give individualized aid to students who do have 
difficulty. 

Yet much of the material one sees is inadequate for students 
in need of aid. The only help may be simply to tell students 
that they made errors, often In terms rather discouraging to the 
student. When "help" Is offered, often it bears the guise of 
brief verbal advice, nay, repeating a definition. Interactive 
help Rcquoncns, bringing users to understand the material and 
taking the time to determine If they do understand the material, 
are. seldom available. Unfortunately, the typical help in s 
screen full of text, with no Interaction, and details repeated , 
Siat have already been stressed. "One level" help will be , 
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inadequate for many students? some may require quite different 
help in order facilitate learning. 

The problem of inadequate aid for the learner In difficulty 
1e related to a larger problem, the lack of Individualization of 
the learning material, Different students learn best in 
different ways; . Good computer based learning material assumes 
this, and provides a wide variety of learning sequences and 
styles, However, much current computer material provides only a 
single learning path, forcing everyone through fhe same approach 
in a lecture-like fashion. 

I might summarize many of these bad practices in content by 
saying that much current comptuer based learning material devotes 
very little time-far too little— to considering the pedagogical 
details of the material, Much of it gives the impression that a 
quick pedagogical idea was generated, and then most oE the energy 
went into the coding. Serious pedagogical material deraands 
serious attention to the curriculum details, with or without 
computers. 

Ijmll ^ Cpnaldera tlonn 

J\ dbvflloper of wtnf|>m*r muM loAffilivj uUrUI soaiAtlpR 
w*y \\m had exparUnud in tuimitflv* ourMujlim dmlnpmont in 
other media; others are now dovulQpftca without previous 
exprlance. Many workora In this field will Inevitably be people 
who have developed learning material in book form, in lecture 
form, or In other ways. Ilcnc*, it is not too surprising that, 
particularly in their early use of computer banod learning, such 
developers tend to employ modes similar to those which drove 
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their earlier activities. They are often still thinking 
implicitly about books or some other medium. 

Developing a "feel" for a new learning medium, an we must do 
in the case of computers, Is a heroic enterprise. We can give no 
very definite rules, since much of this activity depends on 
experience. But I will Indicate a few of the media factors that 
characerize poor practices with computer based learning material. 

Perhaps the central problem is the failure to recognize the 
potential of the computer as an Interac tive medium, Poor 
material has far too few interactions, too much for students to 
read on the screen before doing anything. Nontrivial interaction 



is the hallmark of good computer based learning material/ 
Although difficult, It Is not Impossible to establish 
criteria for "quality of interaction," a numerical measure, 
giving a rough feeling for. the degree of interaction. One aspect 
of interaction, although by no means the only Issue, is the time 
b etween Interactions for the average user. There may well be an 
occasional situation In which the designer wants this interaction 
time to be long, perhaps as long as seven! minutes. Generally 
ttio avoraqe tl m between Interactions (one interaction ends and 
annthar Iwjtna) should prohabiy be more like 15 to 30 seconds, 
Sometimes we want It to be a good bit less than this. It Is 
characteristic of poor material that the average time between 
Interactions is too long, primarily because of excessive amounts 
of text. 

A model can aid in developing such interactions. One can 
ask what good teachers do when they work Individually or with a 
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very small group of students. In such a situation most teachers 

will not "lecture/ but will work Interactively with the 

students, perhaps by asking questions, Learning must be an 

active process, if It is to be accomilshed efficiently. 

Computers provide an excellent medium tor such interactive 

learning. Readers interested in more details about interaction, 

Including an early 'attempt to provide a numerical measure, may 

consult an earlier papjjr of the author, 3 
V 

One bad practice, common with computers and with older 
learning media, is the inadequacy .1 visual or graphic 
information. Many students are not very skilled with language, 
and will only gradually build up language skills in courses. 
These students often need more visual aids in learning than Is 
typically presented. Since teachers and developers of curriculum 
tiateriai are skilled in hahdling verbal information, they tend to 
use too much of it for many learners. We must insist, in . 
examining curriculum material using computers and other media, on 
the necessity of providing sizable amounts of pictorial learning 
material. This information should be of direct help to the 
learner, not just visual "gimmicks/ We don't want just a set of * 
pretty pictures. 

Another annoying feature of much computer hased learning 
material Is the constant repetition of stock phrases. In both 
• simulations and problem generators a given section of computer 
code may be used over and over. This is often desirable, 
particularly when the material Is embedded in a richer learning 
structure. But how undesirable it is to repeat over and over 
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exactly the same phrase, as students progress through the 
programl This is deplorable because it reduces the material to a 
mechanical procedure, whereas computers have the potential for 
engaging students In friendly dialog. A variety of simple 
tehcnlques can easily avoid this impediment. For example, we can 
choose a phrase randomly from a collection of equally appropriate 
phrases for the particular situation. While this may seem to be 
a minor issue, attention, to K this ban "create material inore 
appeailng to students. 

One media-related difficulty is the practice of solving 
"problems'' associated with the learning maerial through other 
media. For example, my experience in making films discloses a 
tendency to. believe that one should solve learning problem** 

K 

associated with the film by means of associated print material. 
Ye'; with most films, because of logistic or other problems, roost 
students never see this associated printed material; so this way 
of trying to solve the problem Is unsatisfactory. Similarly, a 
good unit of computer based learning material should, in most 
cases, not be highly dependent on print material, video, 
.lectures, or other learning media, 

It is poor practice to solve a podagogi cat. difficulty in 
learning material by resorting to another medium, I hasten to 
say that multimedia material itself, with each learning medium 
playing its full- instructional role, is Important lathe learning 
process. But even when various media are effectively used, tlio 
logistic problems of making certain everyone has all thft 
components at the right time can how difficult, in the real 
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world, 'Niuc, laboutory iwUdfil In the *o\wm fm the first 
through the sixth grades, aa In inch projects a ESS and SCIS, has 
proved to be very difficult to uno Jn police, because teachers 
do not' want to be bothered keeping and maintaining' kits of 
material. In looking at computer based learning material, when I 
raise a pedagogical question and the developers o£ the material 
toll me they deal with that problem In the student notes or 
teacher notes, I have qualms, 

r" 

Finally, one sot of media-related problems seems Important 
trough 1 to discuss In a nparate Motion. I next discuss the 
arranqnmont of material on tha scroaii In both space and time. 

As" with other aapACte of ooqputW based learning, newcomccs 
of tun treat the BCfMli the «w way U»»y I'av* treated 'writing 
surfaces" in older media, Thus, the screen is considered like 
ttie page .of a book or some other print-like format or a 
blackboard. Oten developers do not even know about print 
readability since that Information is not used commonly in 
developing ncint material for learning. t Perhaps the only 
curriculum development group to give idequate attention to the 
research literature on readability is The Open University in the . 
United Kingdom. ^ 

Ir some cases .the computer-driven screen allows capabilities 
beyond those available on a page^ For example, timing, by which 
material can develop in time Jto aid the reader's comprehension of 
^ that material, is not possible on a page of a book; when the 
learner turns the page the new material is all there, print and 
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pictures, Dut the screen nltuatlon being wry different, 
material can develop in time. In other situations approaches 
possible in the book form are not economically desirable. One 
important influence upon a number of issues is that of blank 
space, Blank space in a book increases the cost of book; blank 
< fi^ce on the computer screen is Eree. 

Let mo point to a series of readability tact-ors common to 
both print and screen. Then let's look at some factors unique to 
the screen. Various studies show that those factors hurt 
readability, both in print and on screen, yet they are fairly 
common in print and computer material. The first' is the use of 
very long lines of print. In general, the learner handles 
material better with short lines, although the readability 
results are rather complex. Right just^iflcatin, combined with 
left justification, the common' practice in books often cuts down 
on readability, llypenatlon is a negative factor too; it is 
common practice, usually combined with right justification. 

Keeping natural prases, defined either syntactically or 
semantically, together on a line also helps a learner who is not 
a good reader. While books have followed this practice 
occasionally, such an in the Lelber and Leiber popularizations of 
mathematics and science, It is stitl'extrcmcly rare in print 
material. It is almost equally rare In existing computer based 
learning material, although newer "second generation" learning 
■* material is beginning to show it, At present, to maintain 
phrases on the same line typically requires some human' 

\ 

intervention; perhaps that is why it Is not" frequently * 

- 10 - 

. .123. 



implemented. At leant frrm a syntactical point of view (not a 
semantic point of view), It could bo automatic, 

A number of useful ways to emph asize critical worjj; and 
phrases in computer hmi learning material eeldoni appear in 
available material. 

Some of these techniques for emphasis involve M moving w the 
waterial In some way, either turning it off and on— blinking— or 
oscillating it/ or reversing field at some rate. Other 
techniques for emphasizing words and phrases will Involve tuning. 
Thus, one can pause a suitable length of time before and after a 
grammatical entity needing emphasis, just as one does in 
speaking. We can vary thfc rate of textual ouput for such 
entities, a topic we shall il lacuna later, Tl mi nq can also be 
effective in paiifliity a(u<i utu:h punctuation marka an commas, 
semicolons, and pAiUIa. 

One bad praoLiu», almoin uiiiviu aaI with Minting ma ton a Is 
spews text* at the uaer at the fiBtoat ponslble rate, with no 
consideration as to whether' thla in daslrable, In the Educational 
Technology Center for many years we have been examining thin 
question informally and recently more formally. We find that 
very few users other than professionals like the fast rate 
typical of material today. Instead, "normal' users, given a 
choice, use a rate much like the 300 baud rate of communications. 
In interviewing users, we have found that the fast tate is 
disturbing to many. One of our current research projects is 
attempting to gather more detailed information about this issue. 
Our previous experience make it clear to us that text is usually 



displayed too rapidly for most users of the sytein, another very 
common bad practice. 

Although we currently act the initial upcail ui wtfttlala nt 
what we consider the best dofllrahlo rate, very uncommon In 
coup tor baoftd learning material, we alno be I lave it U desirable 
to allow learners to control the rate of text output at any time. 
This raises an issue of an entire set of factors for user control 
of the screen. One of the computer's advantages is that it 
allows such reader control, which is not possible in text 
material. Yet dovolopers commonly ignore this possibility and 
provide no more control than is available in books, Users can 
employ a number of controls with computer based learning 
material. Had practice Is not to allow such controls, 

The handling of Input is often particularly, bad. Long 
Inputs, with much English text, frequently lead to words split at 
the edge of the screen. That is, when the print gets to the edge 
of the screen, the hardware or software shifts part of the word 
to the next line . The edge, however, is not necessarily the 
critical location. One might prefer a won! not to overlap a 
diagram or other text, and so provide controls that prevent such 
overlap, while assuming that a whole word is on one line. 
Strategies for not splitting words are well-known in computing; 
they are used widely in word processing system?, where they are 
typically called "word wrap." Words crossing lines or words 
overlapping with diagrams or others, particularly on long inputs, 
is a frequently seen hut highly undesirable practice. Like many 
bad practices, relatively minor increments in the software can 
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control it, The dlUUlLy \t tUl input ami output provided by 
mont computer language i« qui to Inadaquate, 

Perhaps the worst among a number of bad practices associated 
with student input are the types which allow a program to bomb 
because of a mistake, such as typing a letter instead of a 
number. Such typing mistakes are common, particularly with a 
novice likely to confuse the numbers one and zero with the 
letters T and "0/ Fingers resting lightly on the keyboard can 
"add" characters not intended, For some time special "font ends" 
for novices have provided coding for input which supercedes the 
rather crude input now provided in most computer languages, Many 
current materials do not provide such user-friendly front ends. 

Another useful but rarely available feature for student 
input, Is the timed read, In many situations it Is undesirable to 
let a learner sit indefinitely long before the computer does 
something, Learners may not really understand that they are 
expected to type something at that point or may have no clear 
view of what to type ot may have typed something without pressing 
return. In all these cases timed input can carefully handle the 
situation, \ 

Weak underlying software is likely to generate bad practices 
in computer based learning by poorly designed placement of. text 
on the screen. Most computer languages make the book-like 
assumption that text should always start at the left-hand margin. 
I consider this a very bad practice, seldom desirable. Text 
should appear on the screen by conscious design to best aid 
learners with that particular component of text. 
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Flnally, text should frequently disappear from the screen 
without: erasing information which is still relevant. Text no 
longer applicable to the situation at hand shouldn't be there. 
Modern systems allow ports to the screen to he removed, without 
affecting other parts of the screen. In much current computer 
based learning material at any one time the screen shows a past 
history of what huS happened, often with irrelevant text for the 
current situation. The practice of removing superfluous text, 
concentrating the student's attention on what is currently 
relevant, relates to the frequent use of blank space mentioned 
earlier, 
Othe r Factors 

Other issues should he mentioned which fit cither loosely or 
not at all in the classifications developed. Some have to do 
with tactics used in the materials. A number of tactics are poor 
tactics, but commonly used. 

One, such undesirable tactic is multiple choice questions, 
particularly in on-line quzzes. In some cases this is a 
subconscious choice. People know that multiple choice has 
something to do with computers, so they focus on it when they 
t>tart to develop interactive computer materials. Some computer 
quizzing systems for both off-line and on-line employ primarily 
multiple choice. 

Multiple choice should, I believe, almost never occur in 
computer based Jea;,<ng material for two reasons, First, mutiple 
choice Ifl a poor way of determining what a learner knows. 
Guessing factors play too sizable a role. Further, this mode 0 f 



quizzing fell Into education as a tactic of desperation, nsinq 
relatively crude early technology to handle 1 nrqe numhers of 
students, It has little to recommend it at present. Better 
technology renders it obsolete, It is not needed with a comptuor 
for we can ask a much wider range of questions which avoid 
guouaing factors, and a<\ b.uj .jutdly to reliable 
Information About what auduuu J» anJ L ndt kiuw, 

A number of dapu^ dun ijuwy Unglloh alno fall Into 
thd category of Ui\ puiU^, burnt) £ue hM to uiuUitiLand. For 
example, often jipriUtely after tho question mark, with no 
Intervening spaces, the computer pauses for a reply to a 
question. The display in certainly not characteristic of the way 
we teach people to write English I 

Also common is the widespread use of Y's and N's, or even 
worse l's and O's, in place of "yes" and "no/ While some use of 
single letter mnemonics may be useful it is an undesirable tactic 
to follow in most computer based learning material. Future 
displays, such as those currently in use in Sesame Place, may 
build yes and no keys directly onto the display. At least in 
this cane the full word is there, not some computerized 
abbreviation. t 

As stated earlier, I aspire to urge upon the developers and 
publishers of computer ascd learning material some standards of 
quality. And I wish to suggest to users of computer based 
learning material the possiblity of reasonable judgments 
concerning the quality of that material. Many important positive 
issues in the development of computer based learning material are 
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not tapped. For farther details tho reader can consult a recent 
book by the author, Learning with Computers /' 

I welcome cownts about bad practices from others to add to 
my list, 
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It 1b a pleasure to return to England to give this keynote 
address. During my stay as advisor to the National Development 
Program In Computer Assisted Learning, I ,! , n ded an early CAL 
wetlng at Oxford, 1 met many of you and learned about ongoing 
projects. This trip renews Many old acquaintances. Perhaps I 
haven't been In England frequently enough since then. The 
gentleman at Passport Control at Gatwick commented, 'You have 
been neglecting us-you haven't been here since 1980." I will 
try to do better In the futurel 

This paper has four components. I begin with a aeries of 
assertions about computers In education, some supported later in 
the paper. I then review two examples of computers in education 
developed at the Educational Technology Center. A brief 
Interlude considers the major advantages of the computer In 
learning. Finally I speculate on the future of the computer In 
education, arguing that It will be eventually the dominant 
delivery device for all areas of education. I consider aspects 
of what will happen. 
Some Assertions 

The following statements will not be defended In this paper. 



I wish to make my ponltlon clear on these Issues. 

1. Extremely little good computer based learning material 
• Is available In any country. Much of the material, Including 

commercially published material, Is of very poor quality. 

2. The standards currently in use In computer based 
learning material are extremely low and are In great danger of 
becoming accepted as the standards. 

3. Hany of the materials available are bits and pieces 
rather than coherent collections of learning material. 

<. The computer can be used In many ways In education. 
Philosophical discussions should not rule out certain ways. 
Decisions should be made on pedagogical grounds. 

5. The training of teachers Is a major weakness In our 
current systems. Host of the present ln-eervlce ways of training 
teachers are entirely Inadequate to the task. 

«• In teaching programming at any level-primary school, 
secondary school, or college or □«» v-reity-the major emphasis 
should be on teaching good modern jamming structure. 

, !> It Is very unlikely that good programming courses will 
be taught In BASIC. BASIC should be avoided at all costs. 

8. Authoring languages are useless In generating effective 
computer based learning material. 

Readers who would like to discuss these issues with me further 
are welcome to do so. 

Two Examples of Cpmgite r Based learning 
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To give eonie reality to the notion of using the computer in 
education, 1/ diacusa in thla section two examples developed at 
the Educational Technology Center, described elsewhere in detail. 
1' Introductory Physics Quarter 

About seven years ago we developed the mechanico part of an 
Introductory physics course, based on highly interactive, 

graphic, on-line tests, This course has been used about five 

s 

tinea, with 2,000 students, and! improved^ We are now discussing 
moving a subset of these testa ft personal counters for 
marketing, 

The pedagogical structure of the course is like a Keller 
plan. Personalized System of Instruction, or mastery course. The 
subject matter Is divided into units; students stay with a given 
unit until they perform almost perfectly on tests associated with 
that unit. If a student shows weaknesses on a test, further 
study is required. Eventually the student takes another test 
version covering the same learning objectives. 

Tests are given on-line at the computer. In he typical 
presentation 400 students chose one of the two computer forms, 
which differ in content. About 150 students chose a standard, 
noncomputer variant of the course, The 400 students who choose 
computer versions take about 15,000 on-line exams in ten weeka, 
with the computer generating each exam uniquely, offering 
Immediate and very detailed feedback and help to students and 
doing all the record keeping, Because of the highly relevent 
student assistance, students agreed almost unanimously that the 
quizzes are the main learning material in the course, So we 
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describe this course as quiz-based , 

While this Is a physics course, the technique of structuring 
a computer based course around the quizzes with little additional 
expository material is extremely promising for the future. 
2. S cientific Literacy 

The second set of units were designed several years ago r 
when the EdudaMonal Technology first employed personal 
computer^' These compute based learning dialogs are about one 
and a half to two hours long for a typical user. 

Although the units are In the context of science or 
mathematics, their main objective is not to teach the subject 
matter but rather to bring a wide audience to a broad but deep 
understanding of the nature of scientific activity. Such Issues 
as what constitutes a theory in science and how theories are 
discovered are the main content Issues, though not discussed 
explicitly, 

The programs are divided Into modules, typically eight, each 
about fifteen minutes long, Students can enter any module and 
thus do not need to finish at a single session. Students as 
young as ten years old as well as university students and adult 
learners have used many of the progams successfully, Indeed, we 
know of no other way of teaching these difficult Issues .chat 
potentially nay be as successful, Pull sumaative .evaluation 
based on the objectives has yet to be carried out. 

Repeated extensive formative evaluations have led to 
improving |he units. These took place In schools with children 
about twelve years old through university environments an* In 
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piibllc libraries 

In the public library a computer message invites anyone to 
use it. No one helps students with either the computer or 
aubject utter. The Materials are self-contained, a complete 
learning experience on the computet. Testing in libraries has 
■any advantages, including noting places there the materials sre 
■otlvatlonally weak and in need at Improvement. 

These examples dn not exhaust the ways computers can aid 
learning. They show contrasting mainline Instruction where the 
^ computer plays a very Important role. Other examples, from the 
Questional Technology Center and from other groups Illustrate 
other modes, 

Wh y the Compu te t in Learning 

Wiy Is it that the computer is destined to be such an 
Itaportant factor in human learning at all levels with ill types 
of people? Fundamentally the major factor is Interaction . The 
. fact that the computet can make learning an active as. opposed to 
a passive process implies other Important consequences. 

Hhat does the learner do during the learning process? The 
•odel of learning implicit in present school-bssed education is 
the passive model, information is 'delivered" by (He, teacher or 
by books, and the learner la a passive absorber of that 
information, a spectator. Learning must be active If ideas, 
■etbods, concepts are to be internallied. To be useful to the 
Individual, learning must Involve some activity on the part of 



the learner. A learner, or small group, of learners, working with 
a human tutor, can maintain sect activity. But most of our 
' current learning situations, where many people need to learn and 
limited funds support leernlng institutions, sre passive. 

The computet allows UB to move sway from spectator learning 
st reasonable tpst and to return to Interactive learning for 
twyone. This is not to say that the computer competes well 
»lth'an extremely good tutor. He can, with computers, become 
more interactive than Is' usually possible , 

Once we accept that the computer can make learning 
Interactive, even with large numbers of students, we see some 
consequences. As the computer can query the student, frequently 

can determine what the student knows. 80 the curriculum 
•odules can be adapted to different backgrounds, without any 
.conscious realisation on the part of students. He csn fill in 
Hissing background material or methods. After presentation of 
nee ideas, the program can check using internal quitzes to see if 
the student comprehends. If t, the presentation can be" 
reviewed or new approaches to that material can be offered to the 
student, Thus, learning can become. highly Individually, 
differing for each student In terms of the learning materials and 

ths time, 1 

1 

Another consequence of Interaction Is that we can determine 
the level of Interest of the student. While this Is more 
difficult to do, It is possible In an Interactive environment. 
Hatetlal that Is weak In Interest can be changed, following a 
dlfferent^approach, 



Because of Interaction we have very powerful mechanisms for 
lwprovltig the Material. He can save student responses) these 
responses give us extrenely detailed views of what Is happening 
with students moment by moment. 

Although the computer allows this highly Interactive 
approach, with various benefits following, not all computer based 
learning material la Interactive. He need to develop standards 
for judging the quality of interaction. Often beginning users, 
both students and their teachers, are satisfied with very weak 
forms of Interaction, because it is such an Improvement over 
nonlnteractive learning media. THub, many of the vldeodUc plus 
computer modules produced so far, often by video people, are 
extremely weak with regard to interaction. 
The Future of Education 

In this section I briefly discuss four Important Issues 
concerning the future of education, as affected by computers. 
1- Widespread future Use of Computers in Education 

It seems almost certain that the computer will be used very 
widely In education, not only In formal schools-primary and 
secondary and university—but also in training and In adult 
education. Two issues assure thisi (1) The effectiveness of the 
computer in education and (2) the economics of computers In 
education. The effectiveness follows primarily from Interaction 
and individualization. The economic issues are even more 
obvious. Computers, particularly personal computers, are 
declining rapidly in cost. Furthermore, many companies, 
publishers, computer vendors, and new companies are moving toward 
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developing and marketing computer based learning material. While 
much confusion exists in direction, the total commercial funding 
in this activity Is sizable and growing. These companies 
recognize that a large market will develop, even though at 
present they are very uncertain about the nature of the market 
and uncertain of their role. I refer to these two Issues as the 
"good and bad" reasons for the widespread use of computers In 
education,, ' 

2 - The Future of E ducation Will Not Necessarily Be Desirable 
It should be made clear that at this time It Is not clear 
whether the computer will lead to a better or worse educational 
system than we have today. Like any powerful new technology, 
computers can be used In either desirable or undesirable ways. 

Presently, the very poor computer based learning material 
available Is setting a very low standard. If teachers, 
administrators, and parents continue to accept this low standard, 
It may become the standard. So we may move toward a future with 
very poor ways of using the computer in education, ways which 
lead to undesirable learning. 
3. The Key to a Good Educational future Is an Effective 
Production Sy stem 

The questions of how and where materials will be produced Is 
critical in determining whether computers will aid or retard 
education. If we produce computer baaed learning material with 
care, the same, care which has gone Into major curriculum projects 
such as those at The Open University or the major efforts In the 
United States In the iO's and early 70's, then we can expect the * 
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computer to lead to a better educational system. However, If we 
continue with the current cottage industry structure, with 
teachers producing little odds and ends of material with little 
coherence and little classroom testing, then we will be In 
difficulty. The next five to ten years are the critical period, 

MltuHonal Change Will Be a Critical Part of the Future 
of the Computer In Educat ion 

Given the major/changes which will occur In education, we 
caniwte^eefour educational Institutions to e'.ay the same, 
/Schools md universities will change their nature' In ways which 
are not entirely predictable at present. Much of the pressure 
for these changes will not come from wlthlni educational 
Institutions are conservative and do not change drastically 
without strong external pressures, Including Monetary ones. 
These pressures will become stronger, and so we can expect 
Institutions to change. Distance learning activities will 
Increase In Importance, even In the elementary and secondary 
schools. Mastery or criterion reference modes will become more 
common. The tlmt to move through educational Institutions will 
cnan 9« Md* »JU be more varied thsn at present. There will be 
many other changes. I an predicating that the future of 
education Is a desirable one. The changes for an undesirable 
future are frightening. 

The only way we can move toward better educational systems 
Is by efforts of all of us. The time for this effort Is now. 
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Once upon a time .... 

there was a marvelous invention, almost comparable to the gift of 
fire to humankind, except that it was the product of people's 
minds. It was a revolutionary invention, changing profoundly 
many aspects of human life. Eventually it made possible a new 
way to learn, a way which quickly superceded earlier ways. 

This marvelous new invention was not created for purposes of 
education, so at first teachers ignored it completely. 
Nevertheless teachers around the world heard of the invention, 
and the imaginative ones soon recognized this marvelous 
invention's great potential for education, and they wanted very 
much to use it with their students, eager to benefit from the 
excting new technology.- But for many years such teachers were 
frustrated perhaps because among the great invention's many 
purposes education was of low priority. 

The most innovative teachers, brave souls that they were, 
found it possible to buy the new device, and they set about 
trying to use Lt. These teachers produced some materials for 
their own students, and shared a limited number of these 
materials with others, even though their products were only 
amateur efforts. After all, they were primarily teachers , 



specialists in education; they were not at ease with this new and 
esoteric technology. Yet they were brave pioneers doing their 
best to use the magnificent invention to aid learning. 

As the new invention became better established, there arose 
a group of specialists particularly talented in using the new 
invention. A few of these specialists were also interested in 
education, so they too thought about the invention's role in 
learning. They proposed a few products, but their efforts were 
only minimally useful. These specialists understood little about 
teaching and learning, even though they were experts at the 
wonderful invention. 

Most teachers still avoided producing educational material 
by means of the invention. Some were scornful, saying that the 
new technical development had nothing to do with education. This 
was not surprising, considering that teachers number many 
conservatives among their group who tend to believe that their 
long standing techniques must be the best. Furthermore, teachers 
recognized that soir»e of the early products of the new invention 
were of poor quality. 

Many years elapsed, with occasional educational experiments 
by teachers or specialists." Occasionally a bold experiment 
produced a better educational product, but most products were not 
very useful for students. The potential of the marvelous 
invention remained unimplemented for education. 

Then a few people began to see a better way to proceed. The 
notion evolved that several persons could collaborate in 
developing educational products based upon the marvelous new 
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invention. Teachers could use their special skills to aid the 
learning process, and technological specialists could devote 
themselves to the production. So a production system emerged 
combining the talents of both types of experts. Gradually they 
recognized others who could signficantly contribute to these 
educational materials. Artists provided pictures and other 
elements of design. 

As they produced more educational products using the 
marvelous invention, the world began to accept the idea' that 
these new learning materials could be important in education. 
Companies sprang up, aware that the public would buy this new 
product because of its aid to education. 

The market grew slowly. But eventually the educational 
products based on the wonderful invention appeared everywhere. 
They led to a revolutionary new stage of education, new ways for 
people to learn. 

The marvelous invention was .... 

the printing press. 

f 

Or was it the computer? 
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COMPUTER ASSISTED LEARNING— THE AGE OF REASON 

Alfred Bork 

Educational Technology Center 
University of California 

Irvine, California 92717 • * 

August 15, 1983 

The newer uses of the computer in education have fully 
demonstrated that the computer is an extremely powerful learning 
device, one that has potentials far beyond those available in 
current modes of learning, such as lectures and textbook. The 

r" 

major advantages of the computer, its interactive capabilities, 

• . / 

creating an active learning experience for everybody, and the 

ability, because of interaction, to individualize the learning 

experience to each learner, even with very large numbers of 

learners, are now understood. We also are beginning to 

— under s tand the process for developing computer based learning 

material, the set of steps which will lead from the earliest 

ideas to finished tested material, either for schools, 

universities, for adult education, or for training. 

However, knowing hwto do these things, knowing the 

capabilities, is a far cry from insuring that they will really 

happen. The computer, like any other piece of technology, does 

not inherently contain within itself the reasonable ways to use 

it. In fact, all technologies can be used poorly or they can be 

used well. Most of the learning sequences on computers that are 

i 

being produced now, at all levels, are poor, not at all 
reflecting the full capabilities of the computer. 

Hence, the issue of whether the computer will improve 
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education training is still to be decided. The computer can lead 
to a worse educational and training system than we now have. The 
problem is world wide, demanding cooperation among all countries 
if we are to move toward highly desirable educational futures. 
The transition to computer based learning systems is already upon 
us and can be expected to accelerate rapidly. -In addition to the 
economic benefits resultingln schools and training institutions 
with the use of the computer, more and more apparent as the 
computer drops rapidly in cost, there are also powerful 
commercial reasons involved, large companies that see the 
computer already as an important market. The use of computers in 
education is no longer an academic issue, one that can be debated 
in learnec meetings; rather it is a practical issue must 
address with all speed. 

This paper is structured in three sections. In the first 
section I consider some existing examples of computer based 
learning material. While these are described only briefly in 
this paper, in the talk they will be illustrated in more detail. 
It is difficult to write about computer based learning. In fact, 
the best possibility is for the readers to work with programs 
such as the ones described here. In the second section I 
consider the strategies for producing computer based learning 
material that have been developed at the Educational Technology 
Center. In the third section I consider some of the future 
possibilities with regard to computers in education and training. 
In all these sections, the discussion will of necessity be brief, 
far less -than is deserved by these important topics, considering 
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the t the future of education is the issue that we face. - 
EXAMPLES 

I mention several examples of material that has been 

developed" at the University jof California, Irvine, in the 

/ % 

Educational Technology Center. These examples illustrate quickly 
some of the range of currei t possibilities. 

The first example is beginning quarter of an introductory 
physics course. This course, in its computer form (students have 
a choice of a non-computer version also) consists of about 15 
on-line quizzes. Quizzes are used in a mastery mode; a student 
must perform essentially perfectly on a, given quiz or take 
another version of that quiz over again. . The student who takes a 
quiz 4 times and does not succeed, must see the teacher for 
individualized help before proceeding. 

The learning material is built into the quizzes . For most 

, students, this is all the learning material required for the 

I . - 

| course. The quizzes can often determine exactly what problems 
i the student has. We like to refer to a course of this kind as a 
quifc-based course because, urilike the standard course, it puts 
the focus of the course on quizzes rather than on didactic 
learning material. 

Attention is also given to building up students intuition 
with a "controllable world," allowing the students to "play" with 
mechanical phenomena with workbook sections that discuss exactly 
what should be done. 

We give approximately 15,000 quizzes during ten weeks. A 
student spends an average of about 2 1/2 hours a week on the 
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computer. About 400 students take the computer based course. I 
should also indicate that there are really two computer based 
courses with quite different quizzes; the student is offered a 
choice of content as well as other learner control factors. 

As a second example, I mention a set of materials which were 
developed to aid students in understanding the nature of science. 
These materials, developed more recently for personal computers, 
address important issues of the essence of scientific theories, 
working with a wide variety of students frbm eleven years through 
sixty years, in a variety of situations. They have been widely 
tested, not%only in schools, but also in public libraries and 
science museums. A typical unit in this case lasts about 90 to 
120 minutes. The strategy is to put students in a position where 
they must behave like a scientist, 

A third project I will mention, just in its early stages, 
concerns the question of teacher preparation. As computers 
appear more and more rapidly in schools, massive problems develop 
with regard to how the teachers are to use these new devices. 
The traditional methods of teacher training are inadequate to the 
problem/ given the numbers of teachers and the speed necessary. 
Few teachers in a\iy part of the world are close to centers that 
can give adequate aid. We believe that the only way to tackle 
this problem is through the use of the computer itself. We have 
developed several modules and are seeking funding for others. 
Our hope eventually is to produce a full-scale set of materials 
for teachers covering all aspects of the computer in education, 
modules that can be used by all teachers anywhere the problem is 
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very urgent. 

A PRODUCTION SYSTEM 

In this system I describe a system for producing computer 
based learning material. This system is aimed at producing high 
quality material, at a time when sizable amounts of material must 
be produced, a few years from now. The production system draws 
upon the experiences of other sizable curriculum development 
efforts in recent years, involving a wide variety of other media. 
Thus, the strategies used are, in many ways, similar in overall 
form to those at The Open University in England, the world's 
foremost developers of university level curriculum material. I 
will briefly describe a number of stages. Again, details cannot 
be presented in such a brief time. 

Stage 1 — In any project the initial stage is the decision to 
produce the material which usually involves securing the funding 
also. 

I 

Stage 2 — There must be an overall design of the project, 

describing in some detail each of the modules to be produced. 

\ ; • 
One way of developing this overall design is through a meeting of 

national, or international, experts in the area, including 

teachers with practical Experience. This group, through 

brainstorming and other iWchniques, is asked to come up with the 

descriptors, perhaps a page or so of each of the units to be 

produced. \ 

\ 

Stage 3 — The overall design can now be reviewed and perhaps 
modified, based on these reviews. 

Stage 4 — Detailed pedagogical design follows. Each of the 
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units of the course needs to be fully specified. The result of 
this stage is a script, something like the manuscript of a book, 
a storybook and shooting sciipt of a film, or other descriptions 
of this kind. All the pedagogical details, including the 
complete analysis of student responses, must be given. The 
computer has its own form of script, different than a film or 
book. 

Stage 5 — The script can also then be reviewed by a variety 
of experts, and changes can be made. 

Stage 6 — An important issue in computer based learning is 
the way information is displayed on the screen, the design of the 
screen. The computer is not a book offering different kinds of 
controls in space and time, as to how material is on the screen. 
Some of these are controllable by the user, others by the 
programmers. A competent graphic designer is needed to establish 
the design. 

Stage 7 — The screen design can now be reviewed and possibly 
improved. 

Stage 8 — Before coding takes place in a modern software 
engineering environment, it should be designed. A ^ariety of 
strategies are available for doing this. Design of the code is 
much more likely to produce a code which will be easy to modify, 
maintain, and transport. 

Stage 9 — The code design can then be reviewed. 

Stage 10 — At this stage coding can now take place. Note 
that many stages have preceeded coding. Inadequate structures 

for developing computer based learning material often put the 

i 
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coding process much too early* 

Stage 11 — The code itself can now be reveiwed, to see that 
it actually satisfies the standards of good, modern coding. 
Again, this is important if the material is to be modified in 
later stages. 

Stage 12 — Next the actual running of the program can be 
reviewed internally in the project. At this stage we find that 
if the program does what it is supposed to do. Many changes are 
suggested? the program is now viewable on the screen, and some of 
the ideas which seemed reasonable on paper do not seem desirable 
in the running material. 

Stage 13 — We can now conduct a series of formative 
evaluations, perhaps with increasing numbers of students, and 
revisions following each of these formative evaluation stages. 
This is one of the most important stages in the process. For the 
first time, the actual students are using the material. One of 
the advantages of computer based learning material is that we can 
gather very detailed information about how students are behaving, 
and we can use this information for significant improvements in 
the performance of the learning material. 

Stage 14 --Finally we might conduct, if funds allow, a 
full-scale summative evaluation of the material. But we seldom 
have adequate funds for a full-scale summative evaluation. If 
such an evaluation is conducted, it is probably best to do it 
some years after the materials are in use, so that minor bugs of 
usage can be removed and do not interfere with the major issue of 
the evaluation. 



ERLC 



7 152 



I stress that I do not believe that some of the "short cuts" 
that have been suggested, strategies which try to shorten this 
process, are adequate. The process just described will be seen 
to be, as already noted, similar to good strategies for 
curriculum development in other media, too. In particular I 
strongly recommend again the use of authoring languages and 
systems. While this practice is widespread, and new systems of 
this kind continue to be developed all over the world, the study 
of the products of such systems, as compared to material that is 
produced without authoring systems, suggest that these systems 
seldom lead to first rate material. A group that starts out to 
develop a new authoring capability always "knows" that there was 
something wrong with previous ones but feels that they will be 
able to correct the problems. The difficulty lies not with a 
particular authoring language or system, but with the whole 
notion. 
FUTURE 

The massive changes in our educational systems, implied by 
the computer, in all levels and in all types of educational 
institutions, cannot happen smoothly. Particularly, it seems 
unlikely that there will be any careful planning. What is more 
likely to happen is that. various types of traditional schools and 
institutions will try to preserve their present strategies, 
always fighting rear guard actions to maintain essentially the 
status quo, hoping that the computer will make only minor changes 
and additions to their program. As it becomes more and more 
obvious that the changes will- be major and that the entire 
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clear that as the computer becomes more and more a part of our 
society with powerful enabling tools playing a major role, the 
content of courses will change. Does anyone still need to learn 
. to differentiate and integrate, for example, if it is all to be 
done in the future via computer? As the computer is becoming 
more and more powerful as a learning device, new and more 
powerful computer tools will be created which will transform the 
need for various types of learning. So course content will 
change almost as rapidly as course form during this period of 
time. 

A third factor will be the question of what happens to 
institutions. We have already indicated that thiy will of 
necessity change, even though they are likely to resist change. 
It is easy to take on the form of the new educational strategy 
but harder to deal with the substance. For example, those 
colleges which are currently moving to having large numbers of 
computers available on campus have so far given relatively little 
thought to the issue of how curriculum will be affected. Indeed, 
there seems to be an assumption that "our faculty will do it." 
Yet, frankly, faculty in the United States and many other parts 
of the world, have almost no experience in curriculum 
development. They were not picked as faculty because of that 
experience, and it seems unlikely that they will be able to make 
major changes in their courses by themselves. 

The next issue then, naturally suggested by the previous 
one, is where all the new learning material is going to come 
from, who will pay for it, by what process will it be 
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developed— the process shown here or by some other process, will 
the actual production be under control of commercial forces or 
will the educational establishment, or some parts of governments, 
have at least some say in the matter? We almost lost the 
possibility of using the video technology in any useful way in 
education because of the commercial pressures. It .is not clear 
what is going to happen in this situation at the present time, 
but the stakes are large. The period ahead will be, for all of 
us, a very interesting time, and the future of education depends 
on what happens. 
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FAMILIES, COMPUTERS, LEARNING 
Alfred Bork 

Educational Technology Center 
University of California 
Irvine 5 California 92717 

(714) 856-7403/6945 

November 21, 1983 

The Educational Technology Center is a research and development group 
concerned with computer based learning at the University of California, on 
the Irvine campus. Several years ago, we received a grant from the Fund 
for the Improvement for Post-Secondary Education, within the Department of 
Education, to develop science literacy units for a very wide general 
audience. One of the tenets of the project was that the material should 
be usable in public libraries. We had looked for funding for computer 
based learning material to use in public libraries for several years, and 
finally we were successful. We developed about a dozen hours of material 
for this environment and a similar amount of material on another project 
which also was tested in public libraries, as well as in schools. I will not 
describe the nature of either of these two projects, amply described in 
literature available from the Educational Technology Center."^ 

A number of interesting features were found in our use of computers in 
libraries. Unlike some uses in libraries, we were not providing computers 
for kids to play games or for people to learn to program. Rather, we 
were providing highly interactive computer based learning material. Because 
the material was in the libraries, it had to run completely stand-alone with 
no teachers or others' to help, either with computer or subject matter details. 
Furthermore, the material had to be self-motivating, holding the interest 
of the student, since library visitors leave if they lose interest. 
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One feature that we noted in libraries, which we found unexpectedly, 
was that we often had families working together on running the material. We 
saw many examples, but we did not count the number of such family examples. 
A typical arrangement might involve a mother, a father, and two children. 
Sometimes the children were even too young to read; the older children or the 
parents would patiently ready to them, and then there would be a discussion 
about how to reply. One of the literate members would type in the reply. 
We saw far more of this than we expected; we were not encouraging it in any way 
that we were aware of. 

When I observed families working together at the displays, it was clear 
that a joint learning activity was going on. All the people involved were 
learning something, and part of this learning was coming through the interaction 
of family members with each other, stimulated and motivated by the interaction 
with the computer. Thus, we seemed to be promoting a type of family "togetherness," 
unexpected for us. 

Parents do traditionally help their children with school work. When the 
children have trouble with homework, they will, at least in some families, appeal 
to the parents. But this interaction is often unsatisfactory. The learning 
activity is often, at least in pretense, one that involves only the kids. The 
parents often try to maintain the attitude that they "know, 11 because of their 
ovn school days. 

t 

Projects which have explicityly presented kids with material that their 
parents didn f t know have often run into parental opposition. The most interesting 
example of this kind was with the new mathematics. Few parents had any experience ~ 
with such topics as different number systems, and the more rational explanations 
of long division. Schools quickly realized this, and so some classes were run 
to acquaint parents with the new mathematics. But these were 'only partially 
successful. I would not claim that the unf amiliarity of parents with the new 
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math is the only reason that they new mathematics had such difficulty in the 
American school systems, but it probably played some role. In any case, homework 
is seldom viewed as a joint learning activity by both parents and the children. 

Yet in our society we hear much discussion of life-long learning. Educators 
argue that knowledgable adults must continue to learn all their lives. Adults 
do engage in a variety of learning activities, everything from formal courses, 
such as in a community college, to visits to the public library to obtain books 
in a particular area to learn something about that discipline. As society has 
changed more rapidly, the need for continuing education has become more and 
more pressing. 

I do not know why we saw so many examples of the family learning together 
with our materials in the public library. The materials did address a very wide 
age group, because they were intended to be not only for children but also for 
adult education. The subject which they dealth with, the nature of scientific 
activity, is one that is seldom understood by people of any age in our society, 
even though the society is scientifically and technologically based. 

It is perhaps dangerous to draw conclusions from the few weeks in public 

/ 

library that we observed. It could be that our family usage was something of a 
statistical oddity. But I think rather, that we may have a clue to a new 
educational process for the future. As the computer is used more in our society, 
and as it becomes more accessible in environments in which the family can work 
together in learning, we may see more learning together activities in the family 
life. 

A number of factors are necessary if this is to happen. First, sophisticated 
learning computers must become widely available to the general public, not only 
in such areas as public libraries, but perhaps even more important, within the 
home. Today ? s home computers~~are generally too weak to allow the types of 
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learning that are likely to be important. 

The second ingredient necessary is the development of effective curriculum 
material, learning units which employ the full interactive and individualized 
capabilities of the computer. If these are to be widely used by both the 
parent and the students, the formative evaluation process must work with both 
types of audiences, including family audiences. Public library environments 
provide a very useful way of doing that, based on our past experiences. The 
subject matter of these programs must be serious. That is, if the material 
!, plays down" to young children, it probably will not be that exciting to parents. 
We can write learning material that does appeal effectively to both age levels, 
at least in a very wide variety of subject areas. But this will happen only 
if we have much more serious efforts at curriculum development than we presently 
have. 

A third implication is that, as our educational systems change under the 
impact of computing, more and more of education, at lea&t education concerned 
with knowledge based learning, will take place in the home rather than in the 
school or university environment as it does today. I do not wish to imply that 
schools will cease to exist entirely. Although that is a possibility, I do not 
regard it as likely. Rather, certain components of education will migrate to 
the homes and other components to stay in the school. One example night have the 
kind of split that is seen in George Leonard's Education and Ecstasy , knowledge 
based component and socialization components. If we view Leonard's knowldege 
biased activities in terms of today's and tomorrow's personal computers, it seems 
clear that much of it could take place in the home if adequate materials were 



Finally, I point out that there may be a number of interesting sociological 
consequences of families learning together in addition to the obvious learning 



available. 
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benefits on both sides. Parents will be able to exert a greater degree of 
control over their child's learning environment. I do not, frankly, know how 
desirable this will be. It could lead to social problems, or it could lead to 
a very interesting, pleasant, new form of society. 

I emphasize that the evolution toward families learning together with 
computers is likely to be one that will take considerable time. It will not 
happen tomorrow. But it does present very interesting possibilities for the 
future of education. 
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One aspect of the American educational system which has 
increased greatly in recent years, and shows signs of growing 
even larger, is industrial training. These activities include 
internal training of engineers and other personnel and customer 
training conducted by the company; In large corporations these 
training activities can be very extensive, often covering many 
area? common to schools and universities. 

it, is interesting to speculate why industrial training 
continues to grow at a time when other types of educational 
institutions in our society, such as schools and universities, 
are static or diminishing* One factor in this growth is a 
general unrest about the products of our educational system, 
found in many places; a strong feeling exists that if companies 
are to survive and flourish, they must do more training of their 
own personnel. 

If a company has products specific to that company and 
various people need to use and maintain ^those products, those 
people must be trained. Ir> other cases increasing training 
activity represents underlying unhappiness with traditional 
education. Many companies, explicitly or implicitly, ha;*e simply 
"'jiven up" on the traditional educational system to a startling 
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degree. This 'situation has become common, reflecting the 
widespread distsrust of our educational system. 

The reasons for this increase in training are not the major 
concern of this paper. Rather, I wish to examine the decisions 
involved in beginning to use computer based methods within 
training activities. Relatively little computer based material 
is currently used in training, particularly if one excepts a few 
large companies that have pioneered this area. Most industiral 
training is based on books and lectures, as with traditional 
education. The potential for computer based training is great, 
and almost every sizable industrial training activity is now 
investigating the possibilities of the computer in learning. 

The computer is a powerful device in any type of learning, 
either in training or in more traditional educational activities. 
Good computer based learning material has distinct advantages 
over other ways of delivering education. Hence, we find a 
growing realization that computer based learning will play an 
important part, perhaps even a dominant part, in the training 
systems of the future. This paper "ill not attempt to review the 
considerable advantages of the computer but will assume that 
interest in computer usage already exists. It raises the 
question of how to begin, not the question whether one should 
begin at all. 

The Scale of the Beginning Project 

The first decision that must be reached, if one is to 
develop training material using the computer, is the scale of the 
initial project- That is, how large a computer based project is 
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contemplated. 

The project should not be too large because that would 
involve great costs/ in an uncertain direction for the company. 
Many factors associated with computer based learning can only be 
determined after some experience. Hence, a project which starts 
out with the notion of moving all training immediately to the 
computer is not realistic. 

■ i 

On the other hand, a project can bjb too small to yield/any 
effective information about the use of a learning mode. Thus a 
small segment of a course is likely to/ be of little use in 
understanding the computer and its application in a particular 
training environment. A reasonable place to start is with a 
whole course, probably a brief course. 

Picking a course already in existence is one possibility. 

Another possibility , perhaps betterl given developmental time, 

would be to develop training material related to a new course, 

perhaps training in use or maintenance of a product which is 

currently under development and notlyet released. If a 

* \ 

"conventional" course already existsy the new computer based 
course has a standard of comparison. \ However, anyone who takes 
this direction should understand that comparing two courses with 
different instructional materials is a difficult and expensive 
activity, often beyond the resources of \any expect very large 



concerns. 

FACTORS IN CHOOSING A frlLOT COURSE 

The initial pro 4 *ct, the beginning pilot course, must 
consider several factors. First, the grou?\^is developing 
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experience\about how computers can be effectively used within the 
training group, a major reason for developing the course. 
Second, this project will be important to convince management 
they should proceed further with the development of training 

materials involving the computer. As all good computer learning 

materials are likely to be expensive, having a clear mandate from 
management is important, 

A number of considerations should be kept in mind in 
choosing the pilot course. I describe each briefly. Not all 
will be applicable in each situation. 

1 # The Course is Given in Many Locations . One of the 
advantages of computer based learning material is that, because 
of small personal computers, it can be offered almost anywhere 
electricity is available. Hence, we do not need to bring people 
to a central location, often at considerable expense. Many 
companies have centralized their learning activities, building 
large learning centers in a few places. But often the curriculum 
materials should be available in field offices and sometimes at 
the customer's site. Computer based material lends itself very 
well to such possibilities, so this factor is important when 
using the computer in learning. 

2. An Interactive Learning Experience is Highly Desirable , 
This factor is critical in computer based learning material. No 
other mode of learning, available to many people, provides a 
level of interactivity possible with good computer based learning 
material, except human instructors working with very small 
groups . 
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In computer based learning material students frequently can 
be queried to see what they know and appropriate action taken. 
Indeed, students can answer a question perhaps once every 15 to 
20 seconds in good material. The experience is very different 
for the majority of students from listening to someone talk or 
reading a book. While a few very good learners can create active 
experiences out of these situations, most learners cannot. The 
computer brings interactive learning to almost everyone, but only 
if the course is designed to take this into a« jount. 

3. Individualization is Important . In many situations 
students already know some material. For efficient learning, 
students should be able to work on what they do not know and be 
able to bypass quickly material they do know. In an interactive 
environment, such as that provided by the computer, we can 
determine, by built-in quiz segments, at each stage exactly what 
the learner knows and just what additional help the learner 
needs. So the program can be highly individualized, tailored to 
the needs of a particular learner. This is clearly very 
important when learners come from many backgrounds and is also 
important when taking into account the rate at which different 
students learn. 

4. It i s Desirable to Save Time . In typical training 
situations students who are employees of the company are removed 
from usual routines to undergo the training activity. This means 
they are not functioning as productive individuals within the 
company. Hence, the sooner the training is completed, the 
quicker they can get back to a productive job. In a sense, in 
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most industrial training, the time spent in training represents 
money to the company, so briefer time corresponds to less cost to 
the company. 

Because of the possibilities of individualization the 
computer often cuts down on the amount of time needed for 
training e particular student. Students can proceed at their own 
pace, and experimentation indicates there is often a gain of 
about 25% in saving of student's time. This suggests an 
important factor in choosing the prototype course would be to 
pick a course where the time of the participants is very valuable 
to the company. If tt can be shown that these valuable students 
are spending considerably less time with the computer based 
material than with some other approach, a strong economic 
argument is made for the computer material. 

5. Expensive Equipment is Involved . Another factor 
suggesting a particular prototype course is one learning to use 
the expensive equipment. Thus, the course might consist of 
training people to use an expensive graphics output device. The 
more work time required for students to use the actual equipment, 
the more expensive the course. 

Orten it is possible to simulate on personal computers some 
or all of the components of ^he equipment. If this can be done 
on relatively inexpensive computers, the need for learners to tie 
up expensive equipment for lengthy times can be lessened greatly. 
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